Oct 07 17:01:09.746778 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:01:09.747406 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:01:50.732858 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:02:09.752671 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:02:09.754461 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:03:09.753746 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:03:09.754087 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:03:50.733670 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:04:09.761244 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:04:09.761244 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:05:09.763779 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:05:09.764111 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:05:50.735068 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:06:09.765772 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:06:09.766310 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:07:09.766103 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:07:09.766737 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:07:50.738890 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:08:09.773292 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:08:09.773677 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:09:09.779864 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:09:09.780379 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:09:50.742620 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:10:09.782982 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:10:09.783683 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:11:09.782783 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:11:09.783515 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:11:50.744954 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:12:09.782129 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:12:09.782713 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:13:09.782637 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:13:09.783365 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:13:50.754427 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:14:09.788813 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:14:09.790657 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:15:09.792045 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:15:09.793035 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:15:41.734879 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Create backup started, backup: 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c volume: eb44c84c-78d6-40a0-b17d-2268f817a3d5. Oct 07 17:15:41.739363 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupsv39test-backup-container-1437500637',created_at=2025-10-07T21:15:41Z,data_timestamp=2025-10-07T21:15:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-1519038275',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=1b6bf1a3-3a15-44e7-bd1f-0522797edd0c,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='377832173934482d8d90cdce9f49182a',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='c2b67242126842ce9372df228e07a855',volume_id=eb44c84c-78d6-40a0-b17d-2268f817a3d5) Oct 07 17:15:41.774358 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_377832173934482d8d90cdce9f49182a {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:15:41.774358 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:15:41.774358 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:15:41.775236 np0005474935.novalocal cinder-backup[104204]: INFO oslo.privsep.daemon [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpepiaceb4/privsep.sock'] Oct 07 17:15:41.789923 np0005474935.novalocal sudo[139577]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmpepiaceb4/privsep.sock Oct 07 17:15:42.540457 np0005474935.novalocal cinder-backup[104204]: INFO oslo.privsep.daemon [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Spawned new privsep daemon via rootwrap Oct 07 17:15:42.540773 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Accepted privsep connection to /tmp/tmpepiaceb4/privsep.sock {{(pid=104204) __init__ /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:377}} Oct 07 17:15:42.542090 np0005474935.novalocal cinder-backup[104204]: INFO oslo.privsep.daemon [-] privsep daemon starting Oct 07 17:15:42.542090 np0005474935.novalocal cinder-backup[104204]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Oct 07 17:15:42.542406 np0005474935.novalocal cinder-backup[104204]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_READ_SEARCH|CAP_SYS_ADMIN/CAP_DAC_READ_SEARCH|CAP_SYS_ADMIN/none Oct 07 17:15:42.543017 np0005474935.novalocal cinder-backup[104204]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 139586 Oct 07 17:15:42.545180 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3bd86aa8-9fcb-4d48-9834-ddb1200b8f78]: (2,) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:42.699388 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:42.709882 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:42.710466 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0230776-57ac-4107-b44c-35eb489d5605]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[5d5468d0-48fd-488e-ade4-b7ee31efc379]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:42.715349 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:15:42.717233 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d5468d0-48fd-488e-ade4-b7ee31efc379]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:42.717233 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:15:42.717233 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:15:42.717845 np0005474935.novalocal cinder-backup[104204]: WARNING os_brick.initiator.connectors.nvmeof [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath' Oct 07 17:15:42.719168 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:42.728467 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:42.728735 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[260dc93a-6cbb-4ab4-9d2c-d928d4c69ed3]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:42.730941 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:42.743150 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:42.743150 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebfb63a1-008e-49bf-a67e-213bae48aced]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:42.744243 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7fea4d0-bf44-4fe8-a589-ff1207e3c3c4]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:42.744839 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:42.759747 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:15:42.760184 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:15:42.764836 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:15:42.767011 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fea3:6218', 'fe80::607b:93ff:fe85:142b', 'fe80::fc16:3eff:fe16:8f9e', 'fe80::fc16:3eff:fef2:9d3f', 'fe80::fc16:3eff:feda:bad3', 'fe80::fc16:3eff:fe9e:3bbc', 'fe80::fc16:3eff:fe94:9695', 'fe80::acec:1dff:fe51:d311', 'fe80::fc16:3eff:febe:d9e0'] Oct 07 17:15:42.767395 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:15:42.767973 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:15:42.768663 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] <== get_connector_properties: return (994ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fea3:6218', 'fe80::607b:93ff:fe85:142b', 'fe80::fc16:3eff:fe16:8f9e', 'fe80::fc16:3eff:fef2:9d3f', 'fe80::fc16:3eff:feda:bad3', 'fe80::fc16:3eff:fe9e:3bbc', 'fe80::fc16:3eff:fe94:9695', 'fe80::acec:1dff:fe51:d311', 'fe80::fc16:3eff:febe:d9e0']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:15:46.802104 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:15:46.803913 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'target_lun': 0, 'volume_id': 'eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'auth_method': 'CHAP', 'auth_username': 'X534e2g5R9CZm3nGouKB', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:15:46.804369 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:15:46.805266 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:15:46.805581 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'target_lun': 0, 'volume_id': 'eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'auth_method': 'CHAP', 'auth_username': 'X534e2g5R9CZm3nGouKB', 'auth_password': 'WCg2u3mAeA8Va53p', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:15:46.807241 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:46.818449 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f2cf47d4-1657-4403-8ade-f3a4ac36367f]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:15:46.824515 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:15:46.826419 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2cf47d4-1657-4403-8ade-f3a4ac36367f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:46.827372 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:15:46.828127 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:15:46.921375 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:15:46.922247 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:46.937274 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:46.937684 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[60b738b7-90f0-4ffa-88ab-013692330919]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:46.937950 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:15:46.937950 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:46.938696 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:46.954523 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:46.954991 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b677748-1394-4099-8ea2-406b7237f72b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:46.955444 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5] added Oct 07 17:15:46.955444 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:15:46.955444 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:15:46.955444 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:15:46.955444 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:46.955755 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:15:46.956389 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.035(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:15:46.956738 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:15:47.958572 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:15:47.960071 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:47.977288 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:47.977610 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[37d711f3-38cf-4ef7-8d2b-3f65cb660006]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:15:47.979340 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:47.982942 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:47.995882 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:47.996819 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[98cc7f73-5921-49f0-ae8c-e8fc58914f69]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:47.997223 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:47.998114 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.012923 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.013588 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d59caf5a-eabb-47a0-b4a5-ab94a665f35e]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.013965 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:48.014847 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v X534e2g5R9CZm3nGouKB {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.030364 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v X534e2g5R9CZm3nGouKB" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.031803 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[41f58b5a-3dc6-446f-8be5-a5ef4066db3e]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.032184 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'X534e2g5R9CZm3nGouKB'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:48.033367 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.048172 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.048538 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[694126cf-d762-4327-8748-2ad6204e4cc0]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.049080 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:48.050832 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.068043 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.068225 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f798a551-c47d-40af-a1bb-faa0efeb98a8]: (4, ('tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash)\ntcp: [17] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-0900cc30-251c-4e73-b6c8-c3840a7a0f16 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.068694 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('-m', 'session'): stdout=tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash) Oct 07 17:15:48.068694 np0005474935.novalocal cinder-backup[104204]: tcp: [17] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-0900cc30-251c-4e73-b6c8-c3840a7a0f16 (non-flash) Oct 07 17:15:48.068694 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:15:48.068941 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsi session list stdout=tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash) Oct 07 17:15:48.068941 np0005474935.novalocal cinder-backup[104204]: tcp: [17] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-0900cc30-251c-4e73-b6c8-c3840a7a0f16 (non-flash) Oct 07 17:15:48.068941 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:15:48.069924 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.103828 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --login" returned: 0 in 0.033s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.104071 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[953fdd3b-b389-4e75-8558-1de6e0483767]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.104709 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5, portal: 38.102.83.9,3260] successful. Oct 07 17:15:48.104709 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:48.105679 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.121202 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.121523 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6c25f39-15ac-4ac0-97e7-b24b19e46d1c]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.122118 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:15:48.123148 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.141367 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.141679 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6802a42-f537-4822-bc4d-34e1329f3448]: (4, ('tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash)\ntcp: [17] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-0900cc30-251c-4e73-b6c8-c3840a7a0f16 (non-flash)\ntcp: [18] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.142258 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('-m', 'session'): stdout=tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash) Oct 07 17:15:48.142258 np0005474935.novalocal cinder-backup[104204]: tcp: [17] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-0900cc30-251c-4e73-b6c8-c3840a7a0f16 (non-flash) Oct 07 17:15:48.142258 np0005474935.novalocal cinder-backup[104204]: tcp: [18] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 (non-flash) Oct 07 17:15:48.142258 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:15:48.142641 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsi session list stdout=tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash) Oct 07 17:15:48.142641 np0005474935.novalocal cinder-backup[104204]: tcp: [17] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-0900cc30-251c-4e73-b6c8-c3840a7a0f16 (non-flash) Oct 07 17:15:48.142641 np0005474935.novalocal cinder-backup[104204]: tcp: [18] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 (non-flash) Oct 07 17:15:48.142641 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:15:48.143398 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.315s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:15:48.143791 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:15:48.145983 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] HCTL ('4', '-', '-', 0) found on session 18 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:15:48.146398 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Scanning host 4 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:15:48.149374 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host4/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:48.229902 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host4/scan" returned: 0 in 0.029s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:48.229902 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e10a7b9-24a0-49ac-ac06-039024c1ae73]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:15:48.229902 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Searching for a device in session 18 and hctl ['4', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:15:49.181057 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Searching for a device in session 18 and hctl ['4', '*', '*', 0] yield: sdc {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:15:49.181868 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Connected to sdc using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'target_lun': 0, 'volume_id': 'eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'auth_method': 'CHAP', 'auth_username': 'X534e2g5R9CZm3nGouKB', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:15:49.182409 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.377s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:15:49.182606 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] <== connect_volume: return (2378ms) {'type': 'block', 'scsi_wwn': '3600140520cbdc521ce94eccb6b8695db', 'path': '/dev/sdc'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:15:49.183043 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdc {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:15:49.215484 np0005474935.novalocal sudo[139767]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdc Oct 07 17:15:49.577246 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdc" returned: 0 in 0.394s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:15:49.586056 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:15:49.594546 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] _create_container started, container: tempest-volumesbackupsv39test-backup-container-1437500637,backup: 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:15:49.800205 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] generate_object_name_prefix: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:15:49.806759 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] starting backup of volume: eb44c84c-78d6-40a0-b17d-2268f817a3d5, volume size: 1073741824, object names prefix volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:15:50.261157 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:50.359948 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:50.360122 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:50.492655 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:50.492921 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:50.781641 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:50.918612 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:50.918914 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:51.048782 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:51.049054 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:51.757064 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:51.882122 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:51.882403 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:52.049921 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:52.050175 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:52.443709 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:52.584910 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:52.585046 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:52.704790 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:52.704975 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:53.132360 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:53.267013 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:53.267130 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:53.454801 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:53.454801 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:53.859759 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:54.037814 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:54.037993 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:54.160694 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:54.161297 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:54.458837 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:54.589452 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:54.589592 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:54.755890 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:54.755890 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:54.945569 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:55.051202 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:55.051324 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:55.179888 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:55.183188 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:55.355275 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:55.500489 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:55.500489 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:55.634025 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:55.634169 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:56.189111 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:56.319633 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:56.319866 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:56.452793 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:56.453531 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:57.074558 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:57.211809 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:57.212218 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:57.333657 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:57.333775 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:57.624446 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:57.729754 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:57.729933 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:57.907273 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:57.909418 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:58.204657 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:58.300584 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:58.300584 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:58.436668 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:58.436668 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:58.999295 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:59.112679 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:59.112937 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:59.251023 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:59.251161 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:59.426473 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:15:59.552258 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:15:59.552258 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:15:59.685847 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:15:59.686017 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:15:59.888618 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:00.018077 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:00.018180 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:00.146602 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:00.146748 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:00.338026 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:00.476049 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:00.476049 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:00.643583 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:00.644799 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:00.802951 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:00.912586 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:00.912586 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:01.035064 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:01.035351 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:01.606199 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:01.744094 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:01.744252 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:01.915287 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:01.915478 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:02.441862 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:02.602002 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:02.602155 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:02.745787 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:02.746225 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:02.868978 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:02.920107 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:02.920107 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:03.037282 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup MD5 for volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:03.037282 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:03.044149 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:16:03.057620 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:16:03.058001 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:16:03.068147 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:16:03.068822 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:16:03.084368 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:16:03.084368 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] _write_sha256file started, container name: tempest-volumesbackupsv39test-backup-container-1437500637, sha256file filename: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:16:03.171713 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:16:03.172737 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] _write_metadata started, container name: tempest-volumesbackupsv39test-backup-container-1437500637, metadata filename: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:16:03.226241 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "1b6bf1a3-3a15-44e7-bd1f-0522797edd0c",\n "backup_name": "tempest-VolumesBackupsV39Test-Backup-1519038275",\n "created_at": "2025-10-07 21:15:41+00:00",\n "objects": [\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "eb44c84c-78d6-40a0-b17d-2268f817a3d5",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"eb44c84c-78d6-40a0-b17d-2268f817a3d5\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"c2b67242126842ce9372df228e07a855\\", \\"project_id\\": \\"377832173934482d8d90cdce9f49182a\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:15:40.000000\\", \\"launched_at\\": \\"2025-10-07T21:15:41.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsV39Test-Volume-190359563\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 0\\", \\"provider_auth\\": \\"CHAP X534e2g5R9CZm3nGouKB WCg2u3mAeA8Va53p\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:15:40.000000\\", \\"updated_at\\": \\"2025-10-07T21:15:45.000000\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:16:03.230467 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] backup 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:16:03.452901 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdc {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.487831 np0005474935.novalocal sudo[140235]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdc Oct 07 17:16:03.830737 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdc" returned: 0 in 0.378s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.832469 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'target_lun': 0, 'volume_id': 'eb44c84c-78d6-40a0-b17d-2268f817a3d5', 'auth_method': 'CHAP', 'auth_username': 'X534e2g5R9CZm3nGouKB', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '3600140520cbdc521ce94eccb6b8695db', 'path': '/dev/sdc'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:16:03.832797 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:16:03.833577 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:16:03.835082 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.846724 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.847390 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[907ca251-81ed-4de5-8610-fd20dc9dffa9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:16:03.847869 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:16:03.848152 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:16:03.848152 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:16:03.848152 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:16:03.849180 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:16:03.850029 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.862665 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.862922 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2013814f-37fe-427d-abe5-b6dbf6c6b528]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839\n38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.863903 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.877230 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.877519 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c376fb4-5d66-4b94-876c-cee58626fefa]: (4, ('tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash)\ntcp: [18] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.877944 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('-m', 'session'): stdout=tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash) Oct 07 17:16:03.877944 np0005474935.novalocal cinder-backup[104204]: tcp: [18] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 (non-flash) Oct 07 17:16:03.877944 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:16:03.878202 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsi session list stdout=tcp: [11] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-331490d7-fb85-4598-b499-8127eb88b839 (non-flash) Oct 07 17:16:03.878202 np0005474935.novalocal cinder-backup[104204]: tcp: [18] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 (non-flash) Oct 07 17:16:03.878202 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:16:03.879922 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Resulting device map defaultdict(. at 0x7fe9f6404040>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5'): ({'sdc'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:16:03.880570 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Removing single pathed devices sdc {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:16:03.881448 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.888541 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3ad15068-eb2d-402e-8f44-7a80752b2a35]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:16:03.889901 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:16:03.890671 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ad15068-eb2d-402e-8f44-7a80752b2a35]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.891388 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Flushing IO for device /dev/sdc {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:16:03.892218 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.901425 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.901674 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[83c6e237-2736-41f7-8a8d-e03c62242d58]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.902045 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:16:03.902866 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.955953 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.052s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.956175 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[894854fa-a955-47ad-a667-5179967bee8f]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.958340 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Checking to see if SCSI volumes sdc have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:16:03.958683 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] SCSI volumes sdc have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:16:03.959677 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:16:03.961125 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:03.976234 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:03.976431 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[84cf02bd-993a-4785-afc4-e07db06ffc52]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:03.976824 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:03.978935 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:04.012751 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --logout" returned: 0 in 0.033s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:04.013118 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8a9b8f8-7962-411e-a750-5571c888ae41]: (4, ('Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5, portal: 38.102.83.9,3260]\nLogout of [sid: 18, target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:04.013655 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5, portal: 38.102.83.9,3260] Oct 07 17:16:04.013655 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 18, target: iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5, portal: 38.102.83.9,3260] successful. Oct 07 17:16:04.013655 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:04.014724 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:04.031526 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-eb44c84c-78d6-40a0-b17d-2268f817a3d5 -p 38.102.83.9:3260 --op delete" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:04.031710 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d1b192c-205c-470d-afef-273d7300abd3]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:04.032059 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:04.032382 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.199s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:16:04.032551 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] <== disconnect_volume: return (199ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:16:07.587094 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-971bd1ec-2a87-45a8-ae04-1df248a7a1fe tempest-VolumesBackupsV39Test-1194639260 None] Create backup finished. backup: 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c. Oct 07 17:16:08.707353 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] Delete backup started, backup: 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c. Oct 07 17:16:08.707353 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_377832173934482d8d90cdce9f49182a {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:16:08.707353 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:16:08.707353 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] delete started, backup: 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c, container: tempest-volumesbackupsv39test-backup-container-1437500637, prefix: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:16:08.753005 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] generated object list: ['volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00001', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00002', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00003', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00004', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00005', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00006', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00007', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00008', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00009', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00010', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00011', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00012', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00013', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00014', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00015', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00016', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00017', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00018', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00019', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00020', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00021', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c_metadata', 'volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:16:08.773408 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00001 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.795096 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00002 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.819079 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00003 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.837923 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00004 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.856916 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00005 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.876234 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00006 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.896012 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00007 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.915590 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00008 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.935193 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00009 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.955438 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00010 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.977769 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00011 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:08.998865 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00012 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.019928 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00013 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.043442 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00014 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.073370 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00015 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.095036 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00016 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.118985 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00017 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.149021 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00018 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.189814 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00019 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.216766 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00020 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.242875 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c-00021 in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.279620 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c_metadata in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.308080 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] deleted object: volume_eb44c84c-78d6-40a0-b17d-2268f817a3d5/20251007211549/az_nova_backup_1b6bf1a3-3a15-44e7-bd1f-0522797edd0c_sha256file in container: tempest-volumesbackupsv39test-backup-container-1437500637. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:16:09.308485 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] delete 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:16:09.313161 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:16:09.347605 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] Created reservations ['840549e5-8ed6-488c-ad2c-a5181d2f48b3', 'cae7966c-151e-489e-8900-02836bae0589'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:16:09.385846 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-78f49322-d40a-4484-8449-aa338b09fd74 tempest-VolumesBackupsV39Test-1194639260 None] Delete backup finished, backup 1b6bf1a3-3a15-44e7-bd1f-0522797edd0c deleted. Oct 07 17:16:09.798035 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:16:09.798702 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:16:19.385274 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=39,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:16:49.530531 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Create backup started, backup: b290d82d-c1cd-4e47-8cbf-2a9aba1085d4 volume: 81e7d46f-7854-4e18-ac1a-ed038dea783e snapshot: e91ea939-73a0-42a4-b8b1-6a18a36d5a4f. Oct 07 17:16:49.534444 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumessnapshottestjson-backup-container-1803624761',created_at=2025-10-07T21:16:49Z,data_timestamp=2025-10-07T21:16:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-366288263',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=b290d82d-c1cd-4e47-8cbf-2a9aba1085d4,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='92435bbb41b1489aada22e6b7ef31d8e',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id='e91ea939-73a0-42a4-b8b1-6a18a36d5a4f',status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='a97cdd6559484113bdb934f120875405',volume_id=81e7d46f-7854-4e18-ac1a-ed038dea783e) Oct 07 17:16:51.682931 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_92435bbb41b1489aada22e6b7ef31d8e {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:16:51.683441 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:16:51.683730 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:16:51.684970 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:51.694929 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:51.695171 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[db332689-7a85-4735-bb8c-6016d351a396]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[1ef5a61c-c82c-48bd-ae43-4f5f97edd7b7]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:51.696912 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:16:51.697356 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ef5a61c-c82c-48bd-ae43-4f5f97edd7b7]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:51.697846 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:16:51.698274 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:16:51.699093 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:51.711448 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:51.711666 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac87a460-7ea2-488e-b0fd-7e4d71d7f32d]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:51.712546 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:51.729631 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:51.729841 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[fdc5c035-859f-4985-9f0d-bf645f5fa12f]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:51.730828 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[34c38790-6587-4afa-9ca5-403d4a5ab94a]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:51.731352 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:51.754304 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:16:51.754702 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:16:51.756576 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:16:51.757633 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b'] Oct 07 17:16:51.757914 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:16:51.758172 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:16:51.758739 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] <== get_connector_properties: return (74ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:16:55.770948 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:16:55.771756 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c', 'target_lun': 0, 'volume_id': '43b844e0-c393-4cdf-a32e-aff15299f77c', 'auth_method': 'CHAP', 'auth_username': 'aGuMF3VyQz9vkzBeD55C', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:16:55.773781 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:16:55.773781 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.000s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:16:55.773781 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c', 'target_lun': 0, 'volume_id': '43b844e0-c393-4cdf-a32e-aff15299f77c', 'auth_method': 'CHAP', 'auth_username': 'aGuMF3VyQz9vkzBeD55C', 'auth_password': 'hMLmZWEn6XBTAQBb', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:16:55.773781 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:55.783379 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[549b3be9-5b73-4b0d-919a-b6bd0d0d8cc3]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:16:55.785375 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:16:55.787403 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[549b3be9-5b73-4b0d-919a-b6bd0d0d8cc3]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:55.787703 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:16:55.788103 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.000s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:16:55.788281 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:16:55.790363 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:55.803873 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:55.804496 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d18b781f-9366-4822-8f30-e15d978c51a4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:55.804692 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:16:55.804692 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:55.806362 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:55.821800 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:55.825545 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e94efff3-1c27-4a82-b759-0b3460f8236d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:55.826186 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c] added Oct 07 17:16:55.826186 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:16:55.826186 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:16:55.826186 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:16:55.826186 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:55.826739 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:16:55.827580 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.039(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:16:55.828040 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:16:56.829597 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:16:56.830522 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.839818 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.840288 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0057e1f-de4d-4437-80cf-34f01ff7845a]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:16:56.841080 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.842554 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.850430 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.008s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.850752 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[46ba68ee-5b0b-4377-bbbf-3bce4c6f5f39]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.850946 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.851548 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.865185 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.865354 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9701315-8bc9-4cb8-b56f-272e2be921cd]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.865546 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.866022 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.auth.username -v aGuMF3VyQz9vkzBeD55C {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.874007 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.auth.username -v aGuMF3VyQz9vkzBeD55C" returned: 0 in 0.008s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.874212 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e49d5de-1120-4564-b79c-54713f90e0f0]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.874424 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'aGuMF3VyQz9vkzBeD55C'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.874988 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.890425 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.890584 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[88e6eb8a-01e3-45ce-ba52-9b5405588eb6]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.890831 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.891306 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.900350 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.900594 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d18d52e-2ad5-4cfa-b68f-4f27885dbb65]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.900772 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Oct 07 17:16:56.900772 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:16:56.900891 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Oct 07 17:16:56.900891 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:16:56.901020 np0005474935.novalocal cinder-backup[104204]: WARNING os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Oct 07 17:16:56.901020 np0005474935.novalocal cinder-backup[104204]: Oct 07 17:16:56.901505 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.917907 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --login" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.918034 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[656869de-e79a-483a-babe-37f5067d9001]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.918297 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c, portal: 38.102.83.9,3260] successful. Oct 07 17:16:56.918297 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.919172 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.934342 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.934774 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[28873408-b65f-4cc1-b3ce-f929962ab2c1]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.934966 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:16:56.935599 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:56.946150 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:56.946495 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[37ab3004-aeb5-44a9-adee-53ddfc18b696]: (4, ('tcp: [19] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:56.946723 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('-m', 'session'): stdout=tcp: [19] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c (non-flash) Oct 07 17:16:56.946723 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:16:56.946844 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsi session list stdout=tcp: [19] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c (non-flash) Oct 07 17:16:56.946844 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:16:56.947101 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.159s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:16:56.947228 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:16:56.947646 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] HCTL ('2', '-', '-', 0) found on session 19 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:16:56.947769 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:16:56.948400 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:57.005916 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:57.005916 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[64b8af3c-b43b-4231-a21b-4197b4586113]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:16:57.005916 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Searching for a device in session 19 and hctl ['2', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:16:57.971687 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Searching for a device in session 19 and hctl ['2', '*', '*', 0] yield: sda {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:16:57.972531 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c', 'target_lun': 0, 'volume_id': '43b844e0-c393-4cdf-a32e-aff15299f77c', 'auth_method': 'CHAP', 'auth_username': 'aGuMF3VyQz9vkzBeD55C', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:16:57.973631 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.201s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:16:57.973972 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] <== connect_volume: return (2202ms) {'type': 'block', 'scsi_wwn': '36001405d062482ae2e748efb958b84e7', 'path': '/dev/sda'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:16:57.974706 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:16:58.010154 np0005474935.novalocal sudo[141504]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Oct 07 17:16:58.347431 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.373s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:16:58.352602 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:16:58.360037 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] _create_container started, container: tempest-volumessnapshottestjson-backup-container-1803624761,backup: b290d82d-c1cd-4e47-8cbf-2a9aba1085d4. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:16:58.593600 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] generate_object_name_prefix: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4 {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:16:58.601491 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] starting backup of volume: 81e7d46f-7854-4e18-ac1a-ed038dea783e, volume size: 1073741824, object names prefix volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:16:58.907403 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:59.023483 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:59.023891 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:59.142299 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:59.142620 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:16:59.341502 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:16:59.458871 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:16:59.459121 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:16:59.657148 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:16:59.658030 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:00.724483 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:00.847701 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:00.847822 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:00.995880 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:00.996177 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:01.630469 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:01.722919 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:01.723132 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:01.889611 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:01.889968 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:02.139358 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:02.266234 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:02.266540 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:02.410281 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:02.410431 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:02.635853 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:02.782122 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:02.782292 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:02.955922 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:02.956268 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:03.136439 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:03.268859 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:03.268859 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:03.383475 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:03.383475 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:03.649966 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:03.752923 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:03.753100 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:03.908633 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:03.908782 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:04.150105 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:04.262037 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:04.262215 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:04.422846 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:04.422972 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:04.872660 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:04.986096 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:04.986096 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:05.152570 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:05.153014 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:05.366287 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:05.462866 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:05.462972 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:05.602292 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:05.603525 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:05.864082 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:05.982507 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:05.982683 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:06.093708 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:06.093708 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:07.041221 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:07.196657 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:07.197599 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:07.357934 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:07.358141 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:07.827413 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:07.974946 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:07.975212 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:08.127904 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:08.128349 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:08.864997 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:09.118603 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:09.118603 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:09.284199 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:09.284865 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:09.531721 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:09.670025 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:09.670814 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:09.794270 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:17:09.794456 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:17:09.816577 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:09.816822 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:10.050219 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:10.176513 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:10.176629 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:10.366931 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:10.367390 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:10.625725 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:10.762906 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:10.763106 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:10.938822 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:10.939158 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:11.602768 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:11.720459 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:11.720621 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:11.865744 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:11.865917 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:12.081536 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:12.221068 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:12.221195 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:12.355614 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:12.355731 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:12.461871 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:17:12.515375 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:17:12.515375 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:17:12.589857 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup MD5 for volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:17:12.590056 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:17:12.594645 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:17:12.599064 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:17:12.599269 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:17:12.602064 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:17:12.602164 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:17:12.604465 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:17:12.605007 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] _write_sha256file started, container name: tempest-volumessnapshottestjson-backup-container-1803624761, sha256file filename: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:17:12.709857 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:17:12.710619 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] _write_metadata started, container name: tempest-volumessnapshottestjson-backup-container-1803624761, metadata filename: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:17:12.749869 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "b290d82d-c1cd-4e47-8cbf-2a9aba1085d4",\n "backup_name": "tempest-VolumesSnapshotTestJSON-Backup-366288263",\n "created_at": "2025-10-07 21:16:49+00:00",\n "objects": [\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "81e7d46f-7854-4e18-ac1a-ed038dea783e",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"81e7d46f-7854-4e18-ac1a-ed038dea783e\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"a97cdd6559484113bdb934f120875405\\", \\"project_id\\": \\"92435bbb41b1489aada22e6b7ef31d8e\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"available\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:16:47.000000\\", \\"launched_at\\": \\"2025-10-07T21:16:47.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesSnapshotTestJSON-Volume-1946685680\\", \\"display_description\\": null, \\"provider_location\\": null, \\"provider_auth\\": null, \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": null, \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:16:46.000000\\", \\"updated_at\\": \\"2025-10-07T21:16:47.000000\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:17:12.753490 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] backup b290d82d-c1cd-4e47-8cbf-2a9aba1085d4 finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:17:12.918654 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:12.965689 np0005474935.novalocal sudo[141727]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Oct 07 17:17:13.317131 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.398s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.318452 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c', 'target_lun': 0, 'volume_id': '43b844e0-c393-4cdf-a32e-aff15299f77c', 'auth_method': 'CHAP', 'auth_username': 'aGuMF3VyQz9vkzBeD55C', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405d062482ae2e748efb958b84e7', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:17:13.318664 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:17:13.319233 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:17:13.321110 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.338889 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.339087 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b076295-8b76-4aa0-bac4-82e772d8b6b6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:17:13.339496 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:17:13.339880 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:17:13.339880 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:17:13.339880 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:17:13.340310 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:17:13.341006 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.359998 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.360109 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[09505fe7-682c-4216-b734-ee32b3aa229b]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.360810 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.381090 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.381406 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[74139212-545b-417f-8aa0-c86b922b6d8f]: (4, ('tcp: [19] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.381628 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('-m', 'session'): stdout=tcp: [19] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c (non-flash) Oct 07 17:17:13.381628 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:17:13.381745 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsi session list stdout=tcp: [19] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c (non-flash) Oct 07 17:17:13.381745 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:17:13.382358 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Resulting device map defaultdict(. at 0x7fe9f645d620>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c'): ({'sda'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:17:13.384366 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Removing single pathed devices sda {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:17:13.384366 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.393717 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6c00da58-1081-40ab-8dba-5d07384ab0d0]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:17:13.394795 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:17:13.395610 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c00da58-1081-40ab-8dba-5d07384ab0d0]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.395924 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Flushing IO for device /dev/sda {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:17:13.396591 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.403189 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.006s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.403302 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[bff22e39-8bf6-4477-97c5-2e3b60eccda0]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.403536 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:17:13.404254 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.467362 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.059s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.467362 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9a8ff49-43c5-4565-80dd-d2fd3e11ef4b]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.467362 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Checking to see if SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:17:13.467362 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:17:13.467548 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:17:13.470528 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.490447 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.490447 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[713cdb91-ca67-45b9-a581-c217c943e444]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.490447 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:17:13.490957 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.519140 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --logout" returned: 0 in 0.028s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.519284 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[81b57d51-1171-4001-b159-9a16fa94e230]: (4, ('Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c, portal: 38.102.83.9,3260]\nLogout of [sid: 19, target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.519983 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 19, target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c, portal: 38.102.83.9,3260] Oct 07 17:17:13.519983 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 19, target: iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c, portal: 38.102.83.9,3260] successful. Oct 07 17:17:13.519983 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:17:13.521710 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:17:13.535515 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-43b844e0-c393-4cdf-a32e-aff15299f77c -p 38.102.83.9:3260 --op delete" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:17:13.536824 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c2923de-9c88-48ff-acce-c79e865969ae]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:17:13.537202 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:17:13.537975 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.218s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:17:13.537975 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] <== disconnect_volume: return (219ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:17:17.100953 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-cecd31da-7931-47c9-ba88-18ed14c6c1fe tempest-VolumesSnapshotTestJSON-1624713996 None] Create backup finished. backup: b290d82d-c1cd-4e47-8cbf-2a9aba1085d4. Oct 07 17:17:17.421522 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] Delete backup started, backup: b290d82d-c1cd-4e47-8cbf-2a9aba1085d4. Oct 07 17:17:17.424827 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_92435bbb41b1489aada22e6b7ef31d8e {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:17:17.425171 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:17:17.425505 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] delete started, backup: b290d82d-c1cd-4e47-8cbf-2a9aba1085d4, container: tempest-volumessnapshottestjson-backup-container-1803624761, prefix: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:17:17.490037 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] generated object list: ['volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00001', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00002', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00003', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00004', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00005', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00006', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00007', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00008', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00009', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00010', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00011', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00012', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00013', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00014', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00015', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00016', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00017', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00018', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00019', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00020', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00021', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4_metadata', 'volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:17:17.519934 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00001 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.550289 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00002 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.571720 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00003 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.599045 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00004 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.627693 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00005 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.659965 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00006 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.688498 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00007 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.723075 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00008 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.746653 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00009 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.784279 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00010 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.818268 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00011 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.847535 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00012 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.880896 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00013 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.919586 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00014 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.955892 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00015 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:17.987154 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00016 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.011534 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00017 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.039275 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00018 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.063191 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00019 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.085024 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00020 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.104081 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4-00021 in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.134676 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4_metadata in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.158241 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] deleted object: volume_81e7d46f-7854-4e18-ac1a-ed038dea783e/20251007211658/az_nova_backup_b290d82d-c1cd-4e47-8cbf-2a9aba1085d4_sha256file in container: tempest-volumessnapshottestjson-backup-container-1803624761. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:17:18.158432 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] delete b290d82d-c1cd-4e47-8cbf-2a9aba1085d4 finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:17:18.164639 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:17:18.190808 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] Created reservations ['8610fac3-53fa-487f-a8cc-fcae7c1310f2', 'e546907f-658a-4758-a9a9-86cca4a9b4bd'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:17:18.207508 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-9503a4c1-4217-4614-a498-f8330dd96640 tempest-VolumesSnapshotTestJSON-1624713996 None] Delete backup finished, backup b290d82d-c1cd-4e47-8cbf-2a9aba1085d4 deleted. Oct 07 17:17:28.205772 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=41,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:17:50.770883 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:18:09.800855 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:18:09.801470 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:19:09.804091 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:19:09.804668 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:19:50.796496 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:20:09.808379 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:20:09.809016 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:21:09.813767 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:21:09.814131 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:21:50.778912 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:22:06.368984 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Create backup started, backup: 7913b78b-e761-4ac5-8950-60edbb551f74 volume: 72efe848-a7c4-40d4-b274-70306b13257b. Oct 07 17:22:06.371917 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1485704696',created_at=2025-10-07T21:22:06Z,data_timestamp=2025-10-07T21:22:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-679553132',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=7913b78b-e761-4ac5-8950-60edbb551f74,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7124796c9a81406abe531b7e1755ce19',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='380e905f32ef429a8500f965eb48373f',volume_id=72efe848-a7c4-40d4-b274-70306b13257b) Oct 07 17:22:08.017632 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:22:08.018318 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:22:08.018318 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:22:08.019756 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:08.035208 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:08.035507 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d96325d-78bc-4711-8f1a-6aa3343a21b3]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e618e204-3687-4d83-888f-0e27fb3f2f50]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:08.038182 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:22:08.038793 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e618e204-3687-4d83-888f-0e27fb3f2f50]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:08.039505 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:22:08.040006 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:22:08.042096 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:08.056602 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:08.056763 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[751d763f-14f1-44f2-bcb6-df9cbdee9cdb]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:08.057551 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:08.071615 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:08.071804 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c67d958-06b5-43b7-92d9-78f5a6e4e286]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:08.072586 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2840b53d-d6bd-44a7-ac04-e35b21b796ca]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:08.072938 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:08.094156 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:22:08.094574 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:22:08.096935 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:22:08.098349 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe99:7f60', 'fe80::a4f7:abff:feaa:788e'] Oct 07 17:22:08.098619 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:22:08.098866 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:22:08.099434 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (80ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe99:7f60', 'fe80::a4f7:abff:feaa:788e']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:22:12.111483 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:22:12.112584 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa', 'target_lun': 0, 'volume_id': '467c3a69-6a4a-45bb-ba32-64681034f8aa', 'auth_method': 'CHAP', 'auth_username': 'PjRCCWtfD65CGrUH4kPF', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:22:12.112918 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:22:12.113678 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:22:12.113971 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa', 'target_lun': 0, 'volume_id': '467c3a69-6a4a-45bb-ba32-64681034f8aa', 'auth_method': 'CHAP', 'auth_username': 'PjRCCWtfD65CGrUH4kPF', 'auth_password': 'HrGu2qN89hgB5nZ9', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:22:12.115513 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:12.123131 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c11341ab-4032-4d1f-b532-1f01947f4385]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:22:12.125184 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:22:12.126733 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[c11341ab-4032-4d1f-b532-1f01947f4385]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:12.127687 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:22:12.128365 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:22:12.128731 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:22:12.129678 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:12.139751 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260" returned: 21 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:12.140148 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[091d6eb5-5a23-480a-8336-b79e8248ccd1]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:12.140682 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:22:12.140682 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:12.141615 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:12.151533 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:12.151802 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f47f19cc-7167-4d00-995f-f9a16863dabf]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:12.152239 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa] added Oct 07 17:22:12.152239 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:22:12.152239 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:22:12.152239 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:22:12.152239 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:12.152543 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:22:12.153176 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.024(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:22:12.153564 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:22:13.155134 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:22:13.156527 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.171547 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.171986 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[57096535-44dd-4b73-b4a3-3bf797996941]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:22:13.173944 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.177227 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.191709 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.192048 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8ae9b88-59ca-41cc-a15a-99051999d908]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.192862 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.193987 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.211319 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.211687 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[89a83f9a-fc9d-4173-b109-a95990f12b99]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.212274 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.213485 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.auth.username -v PjRCCWtfD65CGrUH4kPF {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.231792 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.auth.username -v PjRCCWtfD65CGrUH4kPF" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.232208 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[4bdf0174-f69b-44ba-b5e6-2d26601d8b0c]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.232806 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PjRCCWtfD65CGrUH4kPF'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.234440 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.252024 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.252295 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8c0d46f-a9bc-467d-bf3a-d1a5f7348adf]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.252948 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.254028 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.271993 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.272172 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6670c868-34ae-4541-8529-be0006b06189]: (4, ('tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.272620 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash) Oct 07 17:22:13.272620 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:22:13.272799 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash) Oct 07 17:22:13.272799 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:22:13.273718 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.305303 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --login" returned: 0 in 0.031s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.305688 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e38762c0-7849-45ec-99f5-d2db034b396c]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.306343 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa, portal: 38.102.83.9,3260] successful. Oct 07 17:22:13.306343 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.307621 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.325076 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.325520 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[58b6e0f0-861a-4bc0-8817-a742390aa5b8]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.326129 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:13.327415 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.347542 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.347955 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab0dac86-e1fe-480c-b690-8fb67f8948e2]: (4, ('tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash)\ntcp: [24] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.348428 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash) Oct 07 17:22:13.348428 np0005474935.novalocal cinder-backup[104204]: tcp: [24] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa (non-flash) Oct 07 17:22:13.348428 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:22:13.348705 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash) Oct 07 17:22:13.348705 np0005474935.novalocal cinder-backup[104204]: tcp: [24] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa (non-flash) Oct 07 17:22:13.348705 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:22:13.349264 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.221s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:22:13.349579 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:22:13.350713 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] HCTL ('3', '-', '-', 0) found on session 24 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:22:13.351011 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:22:13.352170 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:13.420976 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.026s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:13.420976 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b71a18f0-ffc7-4986-b5a6-a8bd6078167c]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:13.420976 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 24 and hctl ['3', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:22:14.383235 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 24 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:22:14.384213 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa', 'target_lun': 0, 'volume_id': '467c3a69-6a4a-45bb-ba32-64681034f8aa', 'auth_method': 'CHAP', 'auth_username': 'PjRCCWtfD65CGrUH4kPF', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:22:14.385389 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.272s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:22:14.385782 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2272ms) {'type': 'block', 'scsi_wwn': '360014052b070efb35844df7a372dcfc4', 'path': '/dev/sdb'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:22:14.386451 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:14.428905 np0005474935.novalocal sudo[144913]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Oct 07 17:22:14.817598 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.430s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:14.828449 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:22:14.833792 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1485704696,backup: 7913b78b-e761-4ac5-8950-60edbb551f74. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:22:15.097897 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] generate_object_name_prefix: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74 {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:22:15.103926 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] starting backup of volume: 72efe848-a7c4-40d4-b274-70306b13257b, volume size: 1073741824, object names prefix volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:22:15.604878 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:15.725366 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:15.725636 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:15.877103 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:15.877103 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:16.397710 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:16.540700 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:16.540700 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:16.726711 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:16.727279 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:17.223914 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:17.360172 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:17.360686 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:17.502119 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:17.502414 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:18.040085 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:18.154968 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:18.155121 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:18.268528 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:18.268764 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:18.800287 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:18.954967 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:18.955232 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:19.105873 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:19.106617 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:19.445423 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:19.631903 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:19.632226 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:19.772159 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:19.772295 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:19.977885 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:20.115786 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:20.115786 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:20.248594 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:20.248758 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:20.748295 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:20.877140 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:20.877319 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:20.996513 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:20.996806 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:21.600093 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:21.733824 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:21.734008 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:21.887975 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:21.888236 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:22.531849 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:22.717413 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:22.717413 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:22.882713 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:22.883006 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:23.317228 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:23.453597 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:23.453816 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:23.606175 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:23.606539 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:24.059557 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:24.187458 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:24.187615 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:24.304702 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:24.304964 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:24.567276 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:24.704100 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:24.704377 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:24.817377 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:24.817591 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:25.118747 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:25.221173 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:25.221354 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:25.333602 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:25.333803 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:25.775009 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:25.870651 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:25.870814 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:25.988168 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:25.988406 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:26.425388 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:26.563919 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:26.564043 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:26.691438 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:26.691591 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:27.166778 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:27.263757 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:27.263941 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:27.376724 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:27.376986 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:27.896764 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:27.993718 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:27.994005 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:28.101702 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:28.101950 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:28.574039 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:28.707694 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:28.707976 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:28.858384 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:28.858384 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:29.100851 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:29.239027 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:29.239184 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:29.374745 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:29.374966 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:29.495546 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:22:29.542465 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:22:29.542714 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:22:29.603989 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:22:29.604215 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:22:29.610726 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:22:29.624121 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:22:29.624594 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:22:29.631192 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:22:29.631599 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:22:29.637651 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:22:29.638119 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1485704696, sha256file filename: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:22:29.737606 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:22:29.738202 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1485704696, metadata filename: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:22:29.773477 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "7913b78b-e761-4ac5-8950-60edbb551f74",\n "backup_name": "tempest-VolumesBackupsTest-Backup-679553132",\n "created_at": "2025-10-07 21:22:06+00:00",\n "objects": [\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "72efe848-a7c4-40d4-b274-70306b13257b",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"72efe848-a7c4-40d4-b274-70306b13257b\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"380e905f32ef429a8500f965eb48373f\\", \\"project_id\\": \\"7124796c9a81406abe531b7e1755ce19\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:21:12.000000\\", \\"launched_at\\": \\"2025-10-07T21:21:13.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1486496098\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b 0\\", \\"provider_auth\\": \\"CHAP jctELZVkCVT8qte75T4W iFjRkgt2jmo3g3nd\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:21:12.000000\\", \\"updated_at\\": \\"2025-10-07T21:22:06.000000\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:22:29.778767 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] backup 7913b78b-e761-4ac5-8950-60edbb551f74 finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:22:29.963936 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.001106 np0005474935.novalocal sudo[144947]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Oct 07 17:22:30.350440 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.386s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.351299 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa', 'target_lun': 0, 'volume_id': '467c3a69-6a4a-45bb-ba32-64681034f8aa', 'auth_method': 'CHAP', 'auth_username': 'PjRCCWtfD65CGrUH4kPF', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014052b070efb35844df7a372dcfc4', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:22:30.352512 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:22:30.353749 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:22:30.356014 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.365595 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.365880 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bf76718-38c9-4796-8cee-dc270e535cde]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:22:30.366474 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:22:30.366926 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:22:30.366926 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:22:30.366926 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:22:30.367461 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:22:30.368552 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.376828 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.008s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.377109 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[69230508-aaaf-4a36-9195-bf640abd5195]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa\n38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.378431 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.401543 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.401792 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[48015e90-798f-430e-aeee-7c414a06ca8e]: (4, ('tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash)\ntcp: [24] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.402489 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash) Oct 07 17:22:30.402489 np0005474935.novalocal cinder-backup[104204]: tcp: [24] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa (non-flash) Oct 07 17:22:30.402489 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:22:30.402764 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [23] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-72efe848-a7c4-40d4-b274-70306b13257b (non-flash) Oct 07 17:22:30.402764 np0005474935.novalocal cinder-backup[104204]: tcp: [24] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa (non-flash) Oct 07 17:22:30.402764 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:22:30.404082 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9f64b5760>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa'): ({'sdb'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:22:30.404526 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sdb {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:22:30.405588 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.413007 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[7605b0c7-7acd-48dd-84f3-6eb965f84aae]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:22:30.415439 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:22:30.417295 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7605b0c7-7acd-48dd-84f3-6eb965f84aae]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.418007 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sdb {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:22:30.418906 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.426835 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.008s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.427295 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec5d4439-0959-43d4-80e1-05af62d60d25]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.427715 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:22:30.428571 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.490464 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.061s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.490712 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f271711-5b17-4704-a140-c7d1fe71fe06]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.491188 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:22:30.491531 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:22:30.492037 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:22:30.493081 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.509441 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.509692 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[13c848a8-4eae-4265-84ce-95b81d87cbb9]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.510365 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:30.511308 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.550370 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --logout" returned: 0 in 0.039s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.550692 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f898497-a74c-442e-a9d1-5b4a96b3e891]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa, portal: 38.102.83.9,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.551260 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa, portal: 38.102.83.9,3260] Oct 07 17:22:30.551260 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa, portal: 38.102.83.9,3260] successful. Oct 07 17:22:30.551260 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:30.552306 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:22:30.570732 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-467c3a69-6a4a-45bb-ba32-64681034f8aa -p 38.102.83.9:3260 --op delete" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:22:30.570870 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b88cc75f-f4de-4ad1-9483-593a261dd411]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:22:30.571157 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:22:30.571575 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.219s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:22:30.571758 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (220ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:22:34.614891 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-dc5a74dc-d968-4bf9-b033-926c325ae20b tempest-VolumesBackupsTest-2000989888 None] Create backup finished. backup: 7913b78b-e761-4ac5-8950-60edbb551f74. Oct 07 17:22:35.297518 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] Delete backup started, backup: 7913b78b-e761-4ac5-8950-60edbb551f74. Oct 07 17:22:35.299281 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:22:35.299648 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:22:35.299980 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] delete started, backup: 7913b78b-e761-4ac5-8950-60edbb551f74, container: tempest-volumesbackupstest-backup-container-1485704696, prefix: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:22:35.370837 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00001', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00002', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00003', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00004', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00005', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00006', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00007', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00008', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00009', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00010', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00011', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00012', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00013', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00014', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00015', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00016', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00017', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00018', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00019', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00020', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00021', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74_metadata', 'volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:22:35.401660 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00001 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.430963 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00002 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.464381 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00003 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.496954 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00004 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.525592 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00005 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.556642 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00006 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.586814 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00007 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.616790 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00008 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.645872 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00009 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.678905 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00010 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.709201 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00011 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.740861 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00012 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.772434 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00013 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.835944 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00014 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.866878 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00015 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.901704 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00016 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.935627 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00017 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.961871 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00018 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:35.990852 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00019 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:36.019978 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00020 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:36.055257 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74-00021 in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:36.086377 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74_metadata in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:36.123380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_72efe848-a7c4-40d4-b274-70306b13257b/20251007212215/az_nova_backup_7913b78b-e761-4ac5-8950-60edbb551f74_sha256file in container: tempest-volumesbackupstest-backup-container-1485704696. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:22:36.123380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] delete 7913b78b-e761-4ac5-8950-60edbb551f74 finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:22:36.127033 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:22:36.148554 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] Created reservations ['bace5d8d-d90d-43cd-a808-77cc1f4b2335', '4c67b682-c211-433d-b9f2-5fe0ff2a4d7a'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:22:36.174042 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-50db5382-0225-4cd4-ac1f-9d4b5f38dc45 tempest-VolumesBackupsTest-2000989888 None] Delete backup finished, backup 7913b78b-e761-4ac5-8950-60edbb551f74 deleted. Oct 07 17:22:46.171048 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=41,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:23:09.820697 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:23:09.821608 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:23:17.084671 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Create backup started, backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b volume: 720f4d8c-455e-4200-a1be-9f34a180c5e3. Oct 07 17:23:17.089884 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1733860864',created_at=2025-10-07T21:23:17Z,data_timestamp=2025-10-07T21:23:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1767932797',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=37f32d88-0e73-4121-8147-3d11dcfd1a4b,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7124796c9a81406abe531b7e1755ce19',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='380e905f32ef429a8500f965eb48373f',volume_id=720f4d8c-455e-4200-a1be-9f34a180c5e3) Oct 07 17:23:17.129219 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:23:17.129532 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:23:17.130058 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:23:17.132761 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:17.144879 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:17.145372 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cfd907b-c31e-42e8-8663-9fc80abf144b]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[73c77d36-d2e3-488f-b448-8faf47fa8a05]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:17.147736 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:23:17.148496 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[73c77d36-d2e3-488f-b448-8faf47fa8a05]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:17.149090 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:23:17.149562 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:23:17.150873 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:17.165588 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:17.166141 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[295e70f6-e39e-46bc-b1f8-aba9c03a82ba]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:17.167265 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:17.183611 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:17.184548 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[06cb5ccc-99dd-4b34-80d1-e9d3cd06b223]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:17.185745 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a43cb3d5-75dd-446d-b7f5-2be52d6691a7]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:17.186189 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:17.206006 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:23:17.206306 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:23:17.208104 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:23:17.208815 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b'] Oct 07 17:23:17.208992 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:23:17.209168 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:23:17.209563 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (78ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:23:21.219514 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:23:21.220825 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3', 'target_lun': 0, 'volume_id': '720f4d8c-455e-4200-a1be-9f34a180c5e3', 'auth_method': 'CHAP', 'auth_username': '3dTx3Ji6voNpJbv8C6wT', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:23:21.221136 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:23:21.221893 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:23:21.222243 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3', 'target_lun': 0, 'volume_id': '720f4d8c-455e-4200-a1be-9f34a180c5e3', 'auth_method': 'CHAP', 'auth_username': '3dTx3Ji6voNpJbv8C6wT', 'auth_password': 'wM2Y9aVSH6RNTDsQ', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:23:21.223630 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:21.233123 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2ecf96a9-7b8d-4a9d-901f-3aa1ca955bbe]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:23:21.235046 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:23:21.236180 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ecf96a9-7b8d-4a9d-901f-3aa1ca955bbe]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:21.237114 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:23:21.237836 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:23:21.238211 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:23:21.239198 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:21.253142 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260" returned: 21 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:21.253423 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0e8cc0c-5779-4bb6-b08b-81a58b2261a3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:21.253891 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:23:21.253891 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:21.254865 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:21.270526 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:21.270780 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ae6b575-91a9-410e-86f1-d89778f0b91a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:21.271218 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3] added Oct 07 17:23:21.271218 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:23:21.271218 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:23:21.271218 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:23:21.271218 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:21.271524 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:23:21.272196 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.034(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:23:21.272583 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:23:22.274786 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:23:22.276673 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.293803 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.294162 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[8958dd8b-0e11-436e-9191-fc6dafd75a42]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:23:22.295821 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.299620 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.315452 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.316473 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[33610525-e414-48a9-b253-78e3f5153a42]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.317265 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.318675 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.337217 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.337516 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[508c7b69-c785-4f60-bc1a-35f59ef97931]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.337939 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.338865 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v 3dTx3Ji6voNpJbv8C6wT {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.355371 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v 3dTx3Ji6voNpJbv8C6wT" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.355632 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[81c9b000-19a6-4160-9832-7b656c22e77e]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.356072 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', '3dTx3Ji6voNpJbv8C6wT'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.357212 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.373964 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.374184 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fefe580-214f-4b20-856e-4e556205729d]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.374823 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.375762 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.392952 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.393171 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[64db07b4-89db-4eac-be52-093863671637]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.393594 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Oct 07 17:23:22.393594 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:22.393843 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Oct 07 17:23:22.393843 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:23:22.394100 np0005474935.novalocal cinder-backup[104204]: WARNING os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Oct 07 17:23:22.394100 np0005474935.novalocal cinder-backup[104204]: Oct 07 17:23:22.394936 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.419838 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --login" returned: 0 in 0.023s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.419838 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[28d47b41-5a28-4a8f-beb4-2bb5366d5ce9]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.419838 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3, portal: 38.102.83.9,3260] successful. Oct 07 17:23:22.419838 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.420378 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.432586 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.432824 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a93171b-2119-4ccf-b111-bb922eba0042]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.433295 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:22.434223 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.446027 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.446223 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[322b35aa-c410-436f-b9d8-805ed51e2623]: (4, ('tcp: [25] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.446698 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [25] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 (non-flash) Oct 07 17:23:22.446698 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:22.446887 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [25] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 (non-flash) Oct 07 17:23:22.446887 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:23:22.447261 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.210s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:23:22.447485 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:23:22.448011 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] HCTL ('2', '-', '-', 0) found on session 25 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:23:22.448183 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:23:22.449106 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:22.541101 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.027s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:22.541101 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd13a698-8512-48b5-895b-f3eff75bfcda]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:22.541101 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 25 and hctl ['2', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:23:23.480483 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 25 and hctl ['2', '*', '*', 0] yield: sda {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:23:23.481367 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3', 'target_lun': 0, 'volume_id': '720f4d8c-455e-4200-a1be-9f34a180c5e3', 'auth_method': 'CHAP', 'auth_username': '3dTx3Ji6voNpJbv8C6wT', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:23:23.482176 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.260s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:23:23.482582 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2261ms) {'type': 'block', 'scsi_wwn': '36001405a13ea489988c430fbd57cb3c2', 'path': '/dev/sda'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:23:23.483160 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:23.522535 np0005474935.novalocal sudo[145396]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Oct 07 17:23:23.871756 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.386s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:23.881080 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:23:23.887783 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1733860864,backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:23:23.947445 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] generate_object_name_prefix: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:23:23.950439 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] starting backup of volume: 720f4d8c-455e-4200-a1be-9f34a180c5e3, volume size: 1073741824, object names prefix volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:23:24.771731 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:25.356427 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 19857195 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:25.356707 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:25.700918 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001: 82571f7121cb9475563edde615e4e4df {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:25.701154 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:26.051430 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:26.186009 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:26.186285 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:26.317986 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:26.317986 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:26.993848 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:27.137130 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 1362431 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:27.137922 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:27.285697 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003: a35891500449df17e893f84bd88472fb {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:27.285884 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:27.789465 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:27.885496 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:27.885496 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:28.066023 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:28.066164 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:28.345856 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:28.454610 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:28.454860 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:28.613089 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:28.613261 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:29.138551 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:29.273223 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:29.273480 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:29.414306 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:29.415002 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:29.717588 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:29.853270 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:29.853563 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:30.003162 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:30.003514 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:30.458466 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:30.574874 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:30.575157 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:30.691837 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:30.692003 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:31.136762 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:31.265195 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:31.265435 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:31.414444 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:31.414813 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:31.875438 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:32.029879 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:32.030092 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:32.185964 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:32.186193 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:32.755888 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:32.893304 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:32.894102 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:33.010690 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:33.010951 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:33.518787 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:33.655955 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:33.656111 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:33.808838 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:33.809005 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:34.270080 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:34.401722 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:34.402010 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:34.554552 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:34.554792 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:34.828151 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:34.963366 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:34.963809 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:35.118660 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:35.118934 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:35.569395 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:35.737772 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:35.738246 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:35.898142 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:35.898227 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:36.355963 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:36.490783 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:36.491029 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:36.643102 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:36.643840 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:36.876171 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:36.974745 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:36.975880 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:37.138321 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:37.138610 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:37.307074 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:37.403264 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:37.403545 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:37.560827 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:37.560990 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:37.786512 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:37.920273 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:37.920539 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:38.073487 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:38.073729 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:38.270912 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:38.406082 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:38.406479 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:38.556443 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:38.556616 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:38.729225 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:23:38.847264 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:23:38.848000 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:23:38.969845 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:23:38.970136 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:23:38.976770 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:23:38.985504 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:23:38.985892 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:23:38.991304 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:23:38.991635 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:23:38.997053 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Oct 07 17:23:38.997564 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1733860864, sha256file filename: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:23:39.097728 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:23:39.099066 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1733860864, metadata filename: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:23:39.138118 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "37f32d88-0e73-4121-8147-3d11dcfd1a4b",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1767932797",\n "created_at": "2025-10-07 21:23:17+00:00",\n "objects": [\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "82571f7121cb9475563edde615e4e4df",\n "offset": 0\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "a35891500449df17e893f84bd88472fb",\n "offset": 104857600\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "720f4d8c-455e-4200-a1be-9f34a180c5e3",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"720f4d8c-455e-4200-a1be-9f34a180c5e3\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"380e905f32ef429a8500f965eb48373f\\", \\"project_id\\": \\"7124796c9a81406abe531b7e1755ce19\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:23:14.000000\\", \\"launched_at\\": \\"2025-10-07T21:23:16.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-787773802\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 0\\", \\"provider_auth\\": \\"CHAP 3dTx3Ji6voNpJbv8C6wT wM2Y9aVSH6RNTDsQ\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": true, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:23:14.000000\\", \\"updated_at\\": \\"2025-10-07T21:23:19.000000\\"}, \\"volume-glance-metadata\\": {\\"hw_rng_model\\": \\"virtio\\", \\"owner_specified.openstack.md5\\": \\"\\", \\"owner_specified.openstack.sha256\\": \\"\\", \\"owner_specified.openstack.object\\": \\"images/cirros-0.6.3-x86_64-disk\\", \\"os_glance_importing_to_stores\\": \\"\\", \\"os_glance_failed_import\\": \\"\\", \\"image_id\\": \\"470af707-0042-467a-bd47-a7aa11a5a05c\\", \\"image_name\\": \\"cirros-0.6.3-x86_64-disk\\", \\"checksum\\": \\"87617e24a5e30cb3b87fda8c0764838f\\", \\"container_format\\": \\"bare\\", \\"disk_format\\": \\"qcow2\\", \\"min_disk\\": \\"0\\", \\"min_ram\\": \\"0\\", \\"size\\": \\"21692416\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:23:39.144260 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] backup 37f32d88-0e73-4121-8147-3d11dcfd1a4b finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:23:39.360681 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.399583 np0005474935.novalocal sudo[145416]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Oct 07 17:23:39.752698 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.388s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.754783 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3', 'target_lun': 0, 'volume_id': '720f4d8c-455e-4200-a1be-9f34a180c5e3', 'auth_method': 'CHAP', 'auth_username': '3dTx3Ji6voNpJbv8C6wT', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405a13ea489988c430fbd57cb3c2', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:23:39.755159 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:23:39.756009 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:23:39.758181 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.773907 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.774169 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1dbf492f-abfe-48d4-915f-934d60fd76c2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:23:39.774753 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:39.775082 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:23:39.775082 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:23:39.775082 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:23:39.775671 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:23:39.776685 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.792131 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.792476 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3dd07a2d-6057-46cd-8b64-964a16f4cd7c]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.793644 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.810808 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.811068 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d470fb6-e42b-48e3-9779-d2aa24b2ac22]: (4, ('tcp: [25] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.811620 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [25] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 (non-flash) Oct 07 17:23:39.811620 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:39.811916 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [25] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 (non-flash) Oct 07 17:23:39.811916 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:23:39.813113 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9d7f50ae0>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3'): ({'sda'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:23:39.813586 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sda {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:23:39.814634 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.825005 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[87aab07b-bb67-4c6f-834b-37f9ad884276]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:23:39.827210 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:23:39.829439 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[87aab07b-bb67-4c6f-834b-37f9ad884276]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.830135 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sda {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:23:39.831142 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.844115 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.844377 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a6c1f0b-1e33-436f-b58f-40e1a9d73e84]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.844936 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:23:39.845989 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.909365 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.062s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.910570 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[155536ef-d13c-4745-88e6-1001c3558260]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.910570 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:23:39.910570 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:23:39.913412 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:23:39.913412 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.929887 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.930127 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1514d819-df69-405b-92d7-244f5f57b6a3]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.930658 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:39.931816 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.971973 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --logout" returned: 0 in 0.039s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.972191 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7b5a74c-057b-4029-820b-69f5e01bbd33]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3, portal: 38.102.83.9,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.972792 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3, portal: 38.102.83.9,3260] Oct 07 17:23:39.972792 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3, portal: 38.102.83.9,3260] successful. Oct 07 17:23:39.972792 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:39.975097 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:39.991166 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 -p 38.102.83.9:3260 --op delete" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:39.991520 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[13e435b6-a512-4990-8a02-3edfa396adef]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:39.992130 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:39.992683 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.237s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:23:39.993057 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (238ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:23:43.515781 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-efbb77be-d659-416b-a21c-802424bb5645 tempest-VolumesBackupsTest-2000989888 None] Create backup finished. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b. Oct 07 17:23:45.112602 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Restore backup started, backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. Oct 07 17:23:45.124454 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:23:45.124928 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:23:45.125583 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:23:45.129058 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:45.141122 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:45.141271 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[75c5bc5b-8a64-4357-9fdc-5e23ab7d7929]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[942b0160-3b58-4aaf-814b-863f0bb7917c]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:45.145268 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:23:45.146193 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[942b0160-3b58-4aaf-814b-863f0bb7917c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:45.146193 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:23:45.146735 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:23:45.149370 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:45.161400 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:45.161646 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cade91f-656f-4689-9ec4-7efaab5d6253]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:45.162720 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:45.177705 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:45.177953 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4586910-3d3e-4e1a-b742-f4786f6b187d]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:45.179284 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e3709ad-89a3-4328-b2f4-b33370b64615]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:45.180004 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:45.198096 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:23:45.198581 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:23:45.201511 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:23:45.202636 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b'] Oct 07 17:23:45.202941 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:23:45.203450 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:23:45.204093 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (77ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:23:49.720108 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:23:49.720954 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'target_lun': 0, 'volume_id': '3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'auth_method': 'CHAP', 'auth_username': '6BiXMAHcW675Esv7f4aL', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:23:49.721173 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:23:49.721666 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:23:49.721829 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'target_lun': 0, 'volume_id': '3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'auth_method': 'CHAP', 'auth_username': '6BiXMAHcW675Esv7f4aL', 'auth_password': 'TaqHCXJ7LMP53dz7', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:23:49.722761 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:49.752199 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[52ea6d3e-f44d-4404-acbb-fe8940d43117]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:23:49.752982 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:23:49.753737 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[52ea6d3e-f44d-4404-acbb-fe8940d43117]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:49.754429 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:23:49.754959 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:23:49.755199 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:23:49.755916 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:49.769788 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260" returned: 21 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:49.769953 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec24b140-f1ce-47e3-a588-91e1e71ce549]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:49.770391 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:23:49.770391 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:49.771077 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:49.785939 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:49.786106 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6730aa7-d4f0-462a-9641-8e080445b9c1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:49.786409 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26] added Oct 07 17:23:49.786409 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:23:49.786409 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:23:49.786409 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:23:49.786409 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:49.786662 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:23:49.787001 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.032(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:23:49.787197 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:23:50.788629 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:23:50.789980 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.805222 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.805554 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[820214b7-858c-476d-9cd4-1a38c4d8034f]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:23:50.807363 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.811069 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.823080 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.823363 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[dae80616-bad0-4559-a962-23e7c2a764e3]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.823911 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.824836 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.835892 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.011s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.836117 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b58af02-1266-472c-9df2-924943824a46]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.836772 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.837742 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v 6BiXMAHcW675Esv7f4aL {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.853085 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v 6BiXMAHcW675Esv7f4aL" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.853369 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f4ece13-daf4-415a-b091-8395714ddcc9]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.853873 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', '6BiXMAHcW675Esv7f4aL'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.855008 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.872027 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.872238 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffc405ff-4964-4572-9712-e0126b004ed7]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.872918 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.873929 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.889234 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.889538 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f984344-83c3-4352-8fd8-09f4d5fe6e7d]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.890020 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Oct 07 17:23:50.890020 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:50.890273 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Oct 07 17:23:50.890273 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:23:50.890556 np0005474935.novalocal cinder-backup[104204]: WARNING os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Oct 07 17:23:50.890556 np0005474935.novalocal cinder-backup[104204]: Oct 07 17:23:50.891688 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.914488 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --login" returned: 0 in 0.022s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.914945 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[343260fb-2695-4e40-be13-5ed4107b1b76]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.916409 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, portal: 38.102.83.9,3260] successful. Oct 07 17:23:50.916409 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.917000 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.933433 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.933571 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[97dbdebf-04d2-4850-a936-9b4c293a0e62]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.933889 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:50.934733 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:50.958396 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:50.958663 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[22917b05-7e5f-42a4-9e8e-c24af966414a]: (4, ('tcp: [26] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:50.958986 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [26] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 (non-flash) Oct 07 17:23:50.958986 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:50.959249 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [26] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 (non-flash) Oct 07 17:23:50.959249 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:23:50.959637 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.205s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:23:50.959808 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:23:50.960380 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] HCTL ('2', '-', '-', 0) found on session 26 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:23:50.960554 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:23:50.961390 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:51.027248 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.024s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:51.027248 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[37a66997-865a-4d97-b1d2-a012b159a8aa]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:51.027248 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 26 and hctl ['2', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:23:51.991945 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 26 and hctl ['2', '*', '*', 0] yield: sda {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:23:51.992798 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'target_lun': 0, 'volume_id': '3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'auth_method': 'CHAP', 'auth_username': '6BiXMAHcW675Esv7f4aL', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:23:51.993696 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.272s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:23:51.994029 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2273ms) {'type': 'block', 'scsi_wwn': '360014051c9d60ccaa8a48c69c9479b6f', 'path': '/dev/sda'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:23:51.994681 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:52.033567 np0005474935.novalocal sudo[145604]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Oct 07 17:23:52.377925 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.383s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:52.378774 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] starting restore of backup volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b container: tempest-volumesbackupstest-backup-container-1733860864, to new volume 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b. {{(pid=104204) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:783}} Oct 07 17:23:52.379215 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1733860864, metadata filename: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_metadata. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] _read_metadata finished. Metadata: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "backup_description": null, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "backup_id": "37f32d88-0e73-4121-8147-3d11dcfd1a4b", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "backup_name": "tempest-VolumesBackupsTest-Backup-1767932797", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "created_at": "2025-10-07 21:23:17+00:00", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "objects": [ Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "82571f7121cb9475563edde615e4e4df", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 0 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 52428800 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "a35891500449df17e893f84bd88472fb", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 104857600 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 157286400 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 209715200 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 262144000 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 314572800 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 367001600 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 419430400 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 471859200 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 524288000 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 576716800 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 629145600 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 681574400 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 734003200 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 786432000 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 838860800 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 891289600 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 943718400 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 996147200 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021": { Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "length": 25165824, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "offset": 1048576000 Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: ], Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "parent_id": null, Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "version": "1.0.0", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_id": "720f4d8c-455e-4200-a1be-9f34a180c5e3", Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"720f4d8c-455e-4200-a1be-9f34a180c5e3\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"380e905f32ef429a8500f965eb48373f\", \"project_id\": \"7124796c9a81406abe531b7e1755ce19\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-10-07T21:23:14.000000\", \"launched_at\": \"2025-10-07T21:23:16.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-787773802\", \"display_description\": null, \"provider_location\": \"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 0\", \"provider_auth\": \"CHAP 3dTx3Ji6voNpJbv8C6wT wM2Y9aVSH6RNTDsQ\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"cb5faf91-e3d4-47d3-abe9-b252347dc403\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": true, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"5541d238-1514-4f23-ae57-3e747aca38c5\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-10-07T21:23:14.000000\", \"updated_at\": \"2025-10-07T21:23:19.000000\"}, \"volume-glance-metadata\": {\"hw_rng_model\": \"virtio\", \"owner_specified.openstack.md5\": \"\", \"owner_specified.openstack.sha256\": \"\", \"owner_specified.openstack.object\": \"images/cirros-0.6.3-x86_64-disk\", \"os_glance_importing_to_stores\": \"\", \"os_glance_failed_import\": \"\", \"image_id\": \"470af707-0042-467a-bd47-a7aa11a5a05c\", \"image_name\": \"cirros-0.6.3-x86_64-disk\", \"checksum\": \"87617e24a5e30cb3b87fda8c0764838f\", \"container_format\": \"bare\", \"disk_format\": \"qcow2\", \"min_disk\": \"0\", \"min_ram\": \"0\", \"size\": \"21692416\"}}" Oct 07 17:23:52.399539 np0005474935.novalocal cinder-backup[104204]: }. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Oct 07 17:23:52.405220 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Restoring backup version 1.0.0 {{(pid=104204) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:796}} Oct 07 17:23:52.405220 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1733860864, metadata filename: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_metadata. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] _read_metadata finished. Metadata: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "backup_description": null, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "backup_id": "37f32d88-0e73-4121-8147-3d11dcfd1a4b", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "backup_name": "tempest-VolumesBackupsTest-Backup-1767932797", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "created_at": "2025-10-07 21:23:17+00:00", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "objects": [ Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "82571f7121cb9475563edde615e4e4df", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 0 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 52428800 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "a35891500449df17e893f84bd88472fb", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 104857600 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 157286400 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 209715200 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 262144000 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 314572800 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 367001600 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 419430400 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 471859200 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 524288000 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 576716800 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 629145600 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 681574400 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 734003200 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 786432000 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 838860800 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 891289600 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 943718400 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 996147200 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021": { Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "length": 25165824, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "offset": 1048576000 Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: ], Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "parent_id": null, Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "version": "1.0.0", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_id": "720f4d8c-455e-4200-a1be-9f34a180c5e3", Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"720f4d8c-455e-4200-a1be-9f34a180c5e3\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"380e905f32ef429a8500f965eb48373f\", \"project_id\": \"7124796c9a81406abe531b7e1755ce19\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-10-07T21:23:14.000000\", \"launched_at\": \"2025-10-07T21:23:16.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-787773802\", \"display_description\": null, \"provider_location\": \"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 iqn.2010-10.org.openstack:volume-720f4d8c-455e-4200-a1be-9f34a180c5e3 0\", \"provider_auth\": \"CHAP 3dTx3Ji6voNpJbv8C6wT wM2Y9aVSH6RNTDsQ\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"cb5faf91-e3d4-47d3-abe9-b252347dc403\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": true, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"5541d238-1514-4f23-ae57-3e747aca38c5\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-10-07T21:23:14.000000\", \"updated_at\": \"2025-10-07T21:23:19.000000\"}, \"volume-glance-metadata\": {\"hw_rng_model\": \"virtio\", \"owner_specified.openstack.md5\": \"\", \"owner_specified.openstack.sha256\": \"\", \"owner_specified.openstack.object\": \"images/cirros-0.6.3-x86_64-disk\", \"os_glance_importing_to_stores\": \"\", \"os_glance_failed_import\": \"\", \"image_id\": \"470af707-0042-467a-bd47-a7aa11a5a05c\", \"image_name\": \"cirros-0.6.3-x86_64-disk\", \"checksum\": \"87617e24a5e30cb3b87fda8c0764838f\", \"container_format\": \"bare\", \"disk_format\": \"qcow2\", \"min_disk\": \"0\", \"min_ram\": \"0\", \"size\": \"21692416\"}}" Oct 07 17:23:52.415994 np0005474935.novalocal cinder-backup[104204]: }. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Oct 07 17:23:52.421377 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] v1 volume backup restore of 37f32d88-0e73-4121-8147-3d11dcfd1a4b started. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:700}} Oct 07 17:23:52.421377 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] metadata_object_names = ['volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021']. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:707}} Oct 07 17:23:52.468380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_metadata', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:23:52.474482 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:23:52.480080 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:52.579839 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:53.281859 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:53.305045 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:53.542972 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:53.563146 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:53.893222 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:53.916296 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:54.145808 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:54.161431 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:54.351637 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:54.368892 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:54.559743 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:54.578206 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:54.766223 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:54.783732 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:55.059085 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:55.076814 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:55.309594 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:55.326213 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:55.555227 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:55.569619 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:55.756427 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:55.769730 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:55.988893 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:56.002115 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:56.190876 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:56.208706 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:56.391547 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:56.406250 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:56.590727 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:56.605211 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:56.809952 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:56.826068 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:57.072989 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:57.091906 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:57.345517 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:57.360401 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:57.547281 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:57.561518 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:57.776970 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, object name: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021, volume: 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:23:57.790049 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:23:57.888501 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] v1 volume backup restore of 37f32d88-0e73-4121-8147-3d11dcfd1a4b finished. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:772}} Oct 07 17:23:57.888658 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Restoring volume base metadata {{(pid=104204) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Oct 07 17:23:57.896040 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] No metadata of type 'volume-metadata' to restore {{(pid=104204) put /opt/stack/cinder/cinder/backup/driver.py:345}} Oct 07 17:23:57.896142 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Restoring volume glance metadata {{(pid=104204) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Oct 07 17:23:57.958372 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] restore 37f32d88-0e73-4121-8147-3d11dcfd1a4b to 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 finished. {{(pid=104204) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:837}} Oct 07 17:23:57.974834 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.014682 np0005474935.novalocal sudo[145629]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Oct 07 17:23:58.350236 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.374s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.351281 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'target_lun': 0, 'volume_id': '3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 'auth_method': 'CHAP', 'auth_username': '6BiXMAHcW675Esv7f4aL', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014051c9d60ccaa8a48c69c9479b6f', 'path': '/dev/sda'}), 'kwargs': {'force': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:23:58.351682 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:23:58.352463 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:23:58.354040 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.368790 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.369085 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f80584f-9e15-4e4c-843c-3ac52b8a2fa6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:23:58.369610 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:58.369906 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:23:58.369906 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:23:58.369906 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:23:58.370534 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:23:58.371922 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.385866 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.386166 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[188aa377-5a51-47c2-b071-908acb1fc426]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.387249 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.402894 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.403157 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6324bef-290d-449f-8676-e6ff2b30a4b0]: (4, ('tcp: [26] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.403606 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [26] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 (non-flash) Oct 07 17:23:58.403606 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:23:58.403886 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [26] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 (non-flash) Oct 07 17:23:58.403886 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:23:58.405163 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9f64b6980>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26'): ({'sda'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:23:58.405698 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sda {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:23:58.406665 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.415381 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0c7b1d97-73de-4d8b-8cc0-86bd03cb5d89]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:23:58.417823 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:23:58.419021 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c7b1d97-73de-4d8b-8cc0-86bd03cb5d89]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.419514 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sda {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:23:58.420584 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.431088 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.431429 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d71f91b-f483-4b6e-b4bb-5df2da1aac53]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.431971 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:23:58.432986 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.498894 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.064s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.499110 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[05ae0a50-aeff-4290-9e21-919652f43e18]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.499863 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:23:58.500215 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:23:58.504688 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:23:58.505506 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.522963 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.523244 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2176d1d-1e00-4ca0-8e9c-9011623ff83d]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.523563 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:58.524348 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.564074 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --logout" returned: 0 in 0.037s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.564342 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[866d1916-75e7-41c5-9bd9-e56f180affa5]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, portal: 38.102.83.9,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.564921 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, portal: 38.102.83.9,3260] Oct 07 17:23:58.564921 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26, portal: 38.102.83.9,3260] successful. Oct 07 17:23:58.564921 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:58.566993 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:23:58.581816 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 -p 38.102.83.9:3260 --op delete" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:23:58.582092 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9b3be88-dd44-4086-a49e-1384452cbe74]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:23:58.582723 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:23:58.583148 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.231s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:23:58.583491 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (231ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:02.105199 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.manager [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Encryption key ID for volume 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26 already matches encryption key ID in backup 37f32d88-0e73-4121-8147-3d11dcfd1a4b. {{(pid=104204) _run_restore /opt/stack/cinder/cinder/backup/manager.py:874}} Oct 07 17:24:02.127817 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-9eacee5f-ccd7-4fca-94e7-80164a9f4075 tempest-VolumesBackupsTest-2000989888 None] Finished restoring backup 37f32d88-0e73-4121-8147-3d11dcfd1a4b to volume 3a2d7a95-17d3-4ee0-8b68-33f7aa6aac26. Oct 07 17:24:04.890677 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] Delete backup started, backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b. Oct 07 17:24:04.894367 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:24:04.894367 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:24:04.894367 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] delete started, backup: 37f32d88-0e73-4121-8147-3d11dcfd1a4b, container: tempest-volumesbackupstest-backup-container-1733860864, prefix: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:24:04.957481 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_metadata', 'volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:24:04.983434 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00001 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.005534 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00002 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.026567 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00003 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.047553 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00004 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.079272 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00005 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.101869 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00006 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.130633 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00007 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.163004 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00008 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.194710 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00009 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.227361 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00010 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.256663 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00011 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.287306 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00012 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.315688 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00013 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.350445 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00014 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.384363 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00015 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.412793 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00016 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.437915 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00017 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.465140 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00018 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.490350 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00019 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.517544 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00020 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.541400 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b-00021 in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.567530 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_metadata in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.599348 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_720f4d8c-455e-4200-a1be-9f34a180c5e3/20251007212323/az_nova_backup_37f32d88-0e73-4121-8147-3d11dcfd1a4b_sha256file in container: tempest-volumesbackupstest-backup-container-1733860864. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:05.599799 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] delete 37f32d88-0e73-4121-8147-3d11dcfd1a4b finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:24:05.603640 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:24:05.620489 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] Created reservations ['de6df134-ca8c-4188-8f17-d55bab7e414f', '4e6d3838-e76b-4119-9e7d-d2635875442a'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:24:05.641224 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-dbf80aae-e4eb-4a2f-8c30-111dc2c79ab5 tempest-VolumesBackupsTest-2000989888 None] Delete backup finished, backup 37f32d88-0e73-4121-8147-3d11dcfd1a4b deleted. Oct 07 17:24:09.707634 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Create backup started, backup: 86183940-9c1b-440a-a9eb-6e047a93f398 volume: f9c09f88-a5c4-40ff-bfb9-267130e25ed2. Oct 07 17:24:09.711351 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1800894867',created_at=2025-10-07T21:24:09Z,data_timestamp=2025-10-07T21:24:09Z,deleted=False,deleted_at=None,display_description='tempest-backup-description-1550563525',display_name='tempest-VolumesBackupsTest-Backup-1141606523',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=86183940-9c1b-440a-a9eb-6e047a93f398,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7124796c9a81406abe531b7e1755ce19',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='380e905f32ef429a8500f965eb48373f',volume_id=f9c09f88-a5c4-40ff-bfb9-267130e25ed2) Oct 07 17:24:09.734351 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:24:09.734489 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:24:09.734742 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:24:09.737016 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:09.748953 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:09.749057 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2daa8aa6-5d37-4315-8f77-0982cabbd892]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[49ad8c30-a376-4a8b-a953-1f85ffadbe8d]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:09.750793 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:24:09.751165 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[49ad8c30-a376-4a8b-a953-1f85ffadbe8d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:09.751243 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:24:09.751428 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:24:09.753955 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:09.761173 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:09.761258 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[354c449a-5fdc-481b-ab06-874c1103074c]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:09.763996 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:09.772447 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:09.772555 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c684b1f-85d2-49b9-9ad0-cac1f166c1d3]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:09.773260 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[81292370-4884-4c50-bc2a-5fa0ff3cf8eb]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:09.773548 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:09.787456 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:24:09.787744 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:24:09.788842 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:24:09.789340 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b'] Oct 07 17:24:09.789548 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:24:09.789548 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:24:09.789802 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (55ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:09.825557 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:24:09.825695 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:24:13.302132 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:24:13.303396 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'target_lun': 0, 'volume_id': 'f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'auth_method': 'CHAP', 'auth_username': 'Q9gwdDbSvC4Dk937C5Br', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:24:13.303754 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:24:13.304541 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:24:13.304747 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'target_lun': 0, 'volume_id': 'f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'auth_method': 'CHAP', 'auth_username': 'Q9gwdDbSvC4Dk937C5Br', 'auth_password': 'YHaejWg9vUrgAD6B', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:24:13.305998 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:13.314539 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2465043a-a697-46ff-8cb9-b61779d10a02]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:24:13.315701 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:24:13.316585 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2465043a-a697-46ff-8cb9-b61779d10a02]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:13.317411 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:24:13.318049 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:24:13.318445 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:24:13.319383 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:13.335083 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:13.335384 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[8361b742-3ab9-4efc-a4e5-3fdc9848b8d1]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:13.335960 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:24:13.335960 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:13.337021 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:13.353489 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:13.353842 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bcce492-4b3d-4dae-bdec-3883e0a88e08]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:13.354304 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2] added Oct 07 17:24:13.354304 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:24:13.354304 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:24:13.354304 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:24:13.354304 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:13.354577 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:24:13.355434 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.037(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:24:13.355608 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:24:14.356826 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:24:14.358426 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.372926 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.373264 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb21b782-47a2-47b7-9de2-6777ce798215]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:24:14.375184 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.379041 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.391865 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.392283 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9803a5b-c2ba-49b3-94c6-41db49342941]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.392825 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.394282 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.409963 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.410238 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0c270f3-9d6b-4271-9266-93814fb1087c]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.410774 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.412011 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v Q9gwdDbSvC4Dk937C5Br {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.427474 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v Q9gwdDbSvC4Dk937C5Br" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.427748 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[61775444-8c82-4439-88bc-99ef14b854d1]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.428214 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'Q9gwdDbSvC4Dk937C5Br'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.429513 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.445291 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.445544 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d971aa0-525b-4750-82f7-84d150bb2243]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.446136 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.447195 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.462927 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.463157 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[63c345cf-0387-4cf3-a8ee-ee370e9fb4a6]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.463642 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Oct 07 17:24:14.463642 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:14.463896 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Oct 07 17:24:14.463896 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:24:14.464145 np0005474935.novalocal cinder-backup[104204]: WARNING os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Oct 07 17:24:14.464145 np0005474935.novalocal cinder-backup[104204]: Oct 07 17:24:14.465364 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.486792 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --login" returned: 0 in 0.021s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.487256 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f930d871-20d9-4d57-ac4c-4dd163be1f87]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.487711 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2, portal: 38.102.83.9,3260] successful. Oct 07 17:24:14.487711 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.488790 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.502201 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.502550 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ae5ac34-3567-4946-8ed3-b9ed08ae8018]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.503058 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:14.504032 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.521478 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.521694 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1257c48-d267-460a-b7fb-04437579db95]: (4, ('tcp: [27] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.522165 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [27] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 (non-flash) Oct 07 17:24:14.522165 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:14.522468 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [27] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 (non-flash) Oct 07 17:24:14.522468 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:24:14.522972 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.205s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:24:14.523239 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:24:14.524210 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] HCTL ('2', '-', '-', 0) found on session 27 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:24:14.524507 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:24:14.525498 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:14.603072 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.035s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:14.603072 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[de735b7f-259e-492b-a81d-fcc6b77dc253]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:14.603072 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 27 and hctl ['2', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:24:15.565183 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 27 and hctl ['2', '*', '*', 0] yield: sda {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:24:15.566275 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'target_lun': 0, 'volume_id': 'f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'auth_method': 'CHAP', 'auth_username': 'Q9gwdDbSvC4Dk937C5Br', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:24:15.567668 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.263s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:24:15.568130 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2264ms) {'type': 'block', 'scsi_wwn': '36001405cb979ba5a27f42bb874097e79', 'path': '/dev/sda'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:15.568919 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:15.613220 np0005474935.novalocal sudo[145925]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Oct 07 17:24:15.963938 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.395s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:15.971262 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:24:15.979572 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1800894867,backup: 86183940-9c1b-440a-a9eb-6e047a93f398. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:24:16.061285 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] generate_object_name_prefix: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398 {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:24:16.066917 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] starting backup of volume: f9c09f88-a5c4-40ff-bfb9-267130e25ed2, volume size: 1073741824, object names prefix volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:24:16.415888 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:16.528893 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:16.528893 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:16.695530 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:16.695530 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:16.696736 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=100,cinder:UPDATE=19,cinder:INSERT=17 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:24:17.160458 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:17.299058 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:17.299312 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:17.446266 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:17.446508 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:18.370247 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:18.504367 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:18.504515 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:18.621448 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:18.621667 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:18.902702 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:19.008991 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:19.009251 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:19.137755 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:19.138201 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:19.608859 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:19.802480 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:19.802744 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:19.955425 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:19.955425 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:20.254645 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:20.392579 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:20.392810 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:20.530947 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:20.531132 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:21.021373 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:21.163464 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:21.163760 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:21.299411 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:21.299411 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:21.779414 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:21.881855 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:21.882139 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:22.015570 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:22.016001 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:22.206753 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:22.379849 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:22.380029 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:22.497034 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:22.497267 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:23.147194 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:23.257550 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:23.257964 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:23.405544 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:23.405720 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:24.126492 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:24.261134 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:24.261578 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:24.397657 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:24.397927 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:24.587217 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:24.724385 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:24.724700 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:24.932200 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:24.932501 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:25.366199 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:25.479268 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:25.479461 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:25.631254 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:25.631433 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:26.156464 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:26.297210 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:26.297481 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:26.447376 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:26.447824 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:27.132074 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:27.269032 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:27.269186 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:27.418173 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:27.418411 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:27.692629 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:27.836136 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:27.836380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:27.987013 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:27.987205 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:28.485147 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:28.620211 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:28.620529 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:28.771271 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:28.771697 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:29.093869 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:29.231198 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:29.231454 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:29.350595 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:29.350754 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:29.699232 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:29.860902 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:29.861616 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:30.065954 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:30.066236 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:30.533271 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:30.631883 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:30.631991 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:30.780854 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:30.781122 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:30.970718 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:24:31.055726 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:24:31.055984 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:24:31.166475 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:24:31.166718 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:24:31.173755 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:24:31.181110 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:24:31.181317 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:24:31.185134 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:121}} Oct 07 17:24:31.185304 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:24:31.188000 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:24:31.188269 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1800894867, sha256file filename: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:24:31.288653 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:24:31.290073 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1800894867, metadata filename: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:24:31.334765 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] _write_metadata finished. Metadata: b'{\n "backup_description": "tempest-backup-description-1550563525",\n "backup_id": "86183940-9c1b-440a-a9eb-6e047a93f398",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1141606523",\n "created_at": "2025-10-07 21:24:09+00:00",\n "objects": [\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "f9c09f88-a5c4-40ff-bfb9-267130e25ed2",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"f9c09f88-a5c4-40ff-bfb9-267130e25ed2\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"380e905f32ef429a8500f965eb48373f\\", \\"project_id\\": \\"7124796c9a81406abe531b7e1755ce19\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:24:08.000000\\", \\"launched_at\\": \\"2025-10-07T21:24:09.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1063165022\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 0\\", \\"provider_auth\\": \\"CHAP Q9gwdDbSvC4Dk937C5Br YHaejWg9vUrgAD6B\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:24:08.000000\\", \\"updated_at\\": \\"2025-10-07T21:24:12.000000\\"}, \\"volume-metadata\\": {\\"vol-meta1\\": \\"value1\\", \\"vol-meta2\\": \\"value2\\", \\"vol-meta3\\": \\"value3\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:24:31.340253 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] backup 86183940-9c1b-440a-a9eb-6e047a93f398 finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:24:31.505360 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:31.526863 np0005474935.novalocal sudo[145955]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Oct 07 17:24:31.848689 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.343s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:31.850514 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'target_lun': 0, 'volume_id': 'f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 'auth_method': 'CHAP', 'auth_username': 'Q9gwdDbSvC4Dk937C5Br', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405cb979ba5a27f42bb874097e79', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:24:31.850956 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:24:31.851861 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:24:31.854290 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:31.868845 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:31.869237 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1e8655c-520e-435d-95d5-bed185a685ed]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:24:31.869939 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:31.870535 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:24:31.870535 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:24:31.870535 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:24:31.871045 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:24:31.873437 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:31.882297 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:31.882764 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9be13927-58c2-4ea8-bf37-bdb00ec2b499]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:31.884025 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:31.901530 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:31.901895 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[01b84afc-b6dc-48cf-bc14-d2fd94b03130]: (4, ('tcp: [27] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:31.902308 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [27] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 (non-flash) Oct 07 17:24:31.902308 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:31.902667 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [27] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 (non-flash) Oct 07 17:24:31.902667 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:24:31.903964 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9f64db380>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2'): ({'sda'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:24:31.904518 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sda {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:24:31.905660 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:31.916449 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f2ad91fc-f1f4-4d70-a516-27a3cbdf8156]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:24:31.918680 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:24:31.920519 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2ad91fc-f1f4-4d70-a516-27a3cbdf8156]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:31.921263 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sda {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:24:31.922458 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:31.933658 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.011s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:31.933963 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[98cb0ce2-8fbf-43dc-8b7c-88d57fb38e55]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:31.934471 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:24:31.935653 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:32.010041 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.073s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:32.010658 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae7c7dd4-9d1e-4d24-8972-1f9544ef3abb]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:32.010936 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:24:32.011233 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:24:32.011728 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:24:32.013055 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:32.029364 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:32.029702 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e30c293-bd2a-4b4f-9b59-eaa09e75345a]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:32.030203 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:32.031522 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:32.064910 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --logout" returned: 0 in 0.033s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:32.065295 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e74e446-a271-4c9e-8192-0853885b206a]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2, portal: 38.102.83.9,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:32.066070 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2, portal: 38.102.83.9,3260] Oct 07 17:24:32.066070 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2, portal: 38.102.83.9,3260] successful. Oct 07 17:24:32.066070 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:32.067368 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:32.083645 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 -p 38.102.83.9:3260 --op delete" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:32.084037 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e1c4d7e-e611-4ad3-b23d-94c6fc59f4f4]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:32.084723 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:32.085259 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.233s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:24:32.085710 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (234ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:35.615262 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3293415c-f24d-4019-8432-0b1995fd81b6 tempest-VolumesBackupsTest-2000989888 None] Create backup finished. backup: 86183940-9c1b-440a-a9eb-6e047a93f398. Oct 07 17:24:37.865133 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Restore backup started, backup: 86183940-9c1b-440a-a9eb-6e047a93f398 volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. Oct 07 17:24:37.877422 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:24:37.877422 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:24:37.877422 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:24:37.878597 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:37.890559 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:37.890865 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d908c8f1-d9dd-4b48-b53c-1b58dcc15b87]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[122ab232-bdd1-4204-8715-22be271e610b]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:37.893924 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:24:37.894816 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[122ab232-bdd1-4204-8715-22be271e610b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:37.895713 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:24:37.896237 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:24:37.897319 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:37.910500 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:37.910793 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b269196b-5aa3-42b9-b33d-b9d144682b4d]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:37.912257 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:37.926519 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:37.926837 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[03d8d5cb-0e54-46fc-8564-de2c670d3459]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:37.928149 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[20a86a22-deaf-450f-96c3-4b22b7f3c1b7]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:37.928773 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:37.949321 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:24:37.949762 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:24:37.952045 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:24:37.953187 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b'] Oct 07 17:24:37.953647 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:24:37.953963 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:24:37.954537 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (76ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:41.974438 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:24:41.975179 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'target_lun': 0, 'volume_id': 'dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'auth_method': 'CHAP', 'auth_username': 'nETx4SGpMWnUPc8CUjQp', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:24:41.976061 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:24:41.976293 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:24:41.976620 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'target_lun': 0, 'volume_id': 'dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'auth_method': 'CHAP', 'auth_username': 'nETx4SGpMWnUPc8CUjQp', 'auth_password': 'VsDHu3jqb5gjYV3W', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:24:41.977996 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:41.987645 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b27dd239-5e16-4f2a-998a-da34b27ef5c2]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:24:41.989846 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:24:41.991722 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b27dd239-5e16-4f2a-998a-da34b27ef5c2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:41.992581 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:24:41.993276 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:24:41.993692 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:24:41.995311 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:42.009639 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260" returned: 21 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:42.009889 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[40403997-a242-4758-b50a-3ecb3b8c14d2]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:42.010459 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:24:42.010459 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:42.011504 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:42.028471 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:42.028840 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe55d856-73ea-4f4b-914e-0ab9ec6b05e0]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:42.029350 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1] added Oct 07 17:24:42.029350 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:24:42.029350 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:24:42.029350 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:24:42.029350 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:42.029651 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:24:42.030348 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.036(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:24:42.030698 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:24:43.032991 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:24:43.034654 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.050872 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.051515 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[69cb53a0-5d15-4147-bee0-a10f3d06da93]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:24:43.053458 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.056848 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.071161 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.071543 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea02d371-c147-4cc1-8289-e0cfbe166e4d]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.072046 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.073126 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.089363 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.089848 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3069cf42-d3c9-46de-afb4-90a6960342fa]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.090498 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.091618 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v nETx4SGpMWnUPc8CUjQp {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.107783 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v nETx4SGpMWnUPc8CUjQp" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.108076 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[200dc040-2c6e-4e11-b2f7-12b3e1af2e0b]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.108581 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'nETx4SGpMWnUPc8CUjQp'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.109893 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.125903 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.126194 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bd18905-3779-45d3-9c1c-b87b57f43c10]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.126862 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.127880 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.144554 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.144843 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[48be9a9b-5fe4-4166-9ee3-2dcebe7cf168]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.145322 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Oct 07 17:24:43.145322 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:43.145679 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Oct 07 17:24:43.145679 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:24:43.145950 np0005474935.novalocal cinder-backup[104204]: WARNING os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Oct 07 17:24:43.145950 np0005474935.novalocal cinder-backup[104204]: Oct 07 17:24:43.147037 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.169962 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --login" returned: 0 in 0.022s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.170262 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b15845f-4047-4516-8e0a-21020f8f0b1e]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.170803 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1, portal: 38.102.83.9,3260] successful. Oct 07 17:24:43.170803 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.171972 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.190646 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.190910 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ff3ff85e-631b-415d-8eac-781004631387]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.191401 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:43.192408 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.209665 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.210124 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7365b407-5e72-4b54-9f28-21e0805e0ac1]: (4, ('tcp: [28] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.210576 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 (non-flash) Oct 07 17:24:43.210576 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:43.210844 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [28] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 (non-flash) Oct 07 17:24:43.210844 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:24:43.211393 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.218s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:24:43.211679 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:24:43.212539 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] HCTL ('2', '-', '-', 0) found on session 28 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:24:43.212815 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:24:43.213944 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:43.282837 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.033s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:43.282837 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[22d3f7fe-e908-4ebd-a4d6-ba95b5a2f845]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:43.282837 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 28 and hctl ['2', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:24:44.250221 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 28 and hctl ['2', '*', '*', 0] yield: sda {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:24:44.251048 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'target_lun': 0, 'volume_id': 'dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'auth_method': 'CHAP', 'auth_username': 'nETx4SGpMWnUPc8CUjQp', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:24:44.251929 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.276s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:24:44.252250 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2276ms) {'type': 'block', 'scsi_wwn': '36001405a66b37664cc24a3fb66965941', 'path': '/dev/sda'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:44.252819 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:44.291854 np0005474935.novalocal sudo[146149]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Oct 07 17:24:44.626647 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.373s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:44.627534 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] starting restore of backup volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398 container: tempest-volumesbackupstest-backup-container-1800894867, to new volume dccf901c-7f89-4c9d-ad67-fb86447eb9d1, backup: 86183940-9c1b-440a-a9eb-6e047a93f398. {{(pid=104204) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:783}} Oct 07 17:24:44.627896 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1800894867, metadata filename: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_metadata. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] _read_metadata finished. Metadata: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "backup_description": "tempest-backup-description-1550563525", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "backup_id": "86183940-9c1b-440a-a9eb-6e047a93f398", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "backup_name": "tempest-VolumesBackupsTest-Backup-1141606523", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "created_at": "2025-10-07 21:24:09+00:00", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "objects": [ Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 0 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 52428800 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 104857600 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 157286400 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 209715200 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 262144000 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 314572800 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 367001600 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 419430400 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 471859200 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 524288000 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 576716800 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 629145600 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 681574400 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 734003200 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 786432000 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 838860800 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 891289600 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 943718400 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 996147200 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021": { Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "length": 25165824, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "offset": 1048576000 Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: ], Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "parent_id": null, Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "version": "1.0.0", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_id": "f9c09f88-a5c4-40ff-bfb9-267130e25ed2", Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"f9c09f88-a5c4-40ff-bfb9-267130e25ed2\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"380e905f32ef429a8500f965eb48373f\", \"project_id\": \"7124796c9a81406abe531b7e1755ce19\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-10-07T21:24:08.000000\", \"launched_at\": \"2025-10-07T21:24:09.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-1063165022\", \"display_description\": null, \"provider_location\": \"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 0\", \"provider_auth\": \"CHAP Q9gwdDbSvC4Dk937C5Br YHaejWg9vUrgAD6B\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"cb5faf91-e3d4-47d3-abe9-b252347dc403\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": false, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"5541d238-1514-4f23-ae57-3e747aca38c5\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-10-07T21:24:08.000000\", \"updated_at\": \"2025-10-07T21:24:12.000000\"}, \"volume-metadata\": {\"vol-meta1\": \"value1\", \"vol-meta2\": \"value2\", \"vol-meta3\": \"value3\"}}" Oct 07 17:24:44.652892 np0005474935.novalocal cinder-backup[104204]: }. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Oct 07 17:24:44.659483 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Restoring backup version 1.0.0 {{(pid=104204) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:796}} Oct 07 17:24:44.659483 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1800894867, metadata filename: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_metadata. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] _read_metadata finished. Metadata: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "backup_description": "tempest-backup-description-1550563525", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "backup_id": "86183940-9c1b-440a-a9eb-6e047a93f398", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "backup_name": "tempest-VolumesBackupsTest-Backup-1141606523", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "created_at": "2025-10-07 21:24:09+00:00", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "objects": [ Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 0 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 52428800 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 104857600 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 157286400 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 209715200 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 262144000 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 314572800 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 367001600 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 419430400 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 471859200 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 524288000 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 576716800 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 629145600 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 681574400 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 734003200 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 786432000 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 838860800 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 891289600 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 943718400 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 52428800, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "25e317773f308e446cc84c503a6d1f85", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 996147200 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021": { Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "compression": "zlib", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "length": 25165824, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "offset": 1048576000 Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: } Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: ], Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "parent_id": null, Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "version": "1.0.0", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_id": "f9c09f88-a5c4-40ff-bfb9-267130e25ed2", Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"f9c09f88-a5c4-40ff-bfb9-267130e25ed2\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"380e905f32ef429a8500f965eb48373f\", \"project_id\": \"7124796c9a81406abe531b7e1755ce19\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-10-07T21:24:08.000000\", \"launched_at\": \"2025-10-07T21:24:09.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-1063165022\", \"display_description\": null, \"provider_location\": \"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 iqn.2010-10.org.openstack:volume-f9c09f88-a5c4-40ff-bfb9-267130e25ed2 0\", \"provider_auth\": \"CHAP Q9gwdDbSvC4Dk937C5Br YHaejWg9vUrgAD6B\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"cb5faf91-e3d4-47d3-abe9-b252347dc403\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": false, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"5541d238-1514-4f23-ae57-3e747aca38c5\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-10-07T21:24:08.000000\", \"updated_at\": \"2025-10-07T21:24:12.000000\"}, \"volume-metadata\": {\"vol-meta1\": \"value1\", \"vol-meta2\": \"value2\", \"vol-meta3\": \"value3\"}}" Oct 07 17:24:44.672626 np0005474935.novalocal cinder-backup[104204]: }. {{(pid=104204) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Oct 07 17:24:44.679045 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] v1 volume backup restore of 86183940-9c1b-440a-a9eb-6e047a93f398 started. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:700}} Oct 07 17:24:44.679045 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] metadata_object_names = ['volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021']. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:707}} Oct 07 17:24:44.731127 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_metadata', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:24:44.736819 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:24:44.742370 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:44.761155 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:44.979160 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:44.994912 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:45.185473 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:45.205014 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:45.393900 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:45.414892 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:45.652058 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:45.672699 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:45.904132 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:45.921014 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:46.184939 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:46.203396 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:46.434819 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:46.454416 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:46.673395 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:46.692968 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:46.880781 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:46.899884 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:47.103454 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:47.121300 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:47.349855 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:47.367992 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:47.624449 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:47.638633 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:47.913467 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:47.928978 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:48.146901 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:48.166248 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:48.370547 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:48.392427 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:48.630302 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:48.647166 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:48.883282 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:48.898838 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:49.131671 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:49.153056 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:49.384752 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:49.404746 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:49.619168 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restoring object. backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, object name: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021, volume: dccf901c-7f89-4c9d-ad67-fb86447eb9d1. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Oct 07 17:24:49.636783 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] decompressing data using zlib algorithm {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Oct 07 17:24:49.779038 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] v1 volume backup restore of 86183940-9c1b-440a-a9eb-6e047a93f398 finished. {{(pid=104204) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:772}} Oct 07 17:24:49.779462 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Restoring volume base metadata {{(pid=104204) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Oct 07 17:24:49.794720 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Restoring volume metadata {{(pid=104204) _restore_vol_meta /opt/stack/cinder/cinder/backup/driver.py:249}} Oct 07 17:24:49.802378 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=104204) put /opt/stack/cinder/cinder/backup/driver.py:345}} Oct 07 17:24:49.802588 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] restore 86183940-9c1b-440a-a9eb-6e047a93f398 to dccf901c-7f89-4c9d-ad67-fb86447eb9d1 finished. {{(pid=104204) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:837}} Oct 07 17:24:49.803030 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:49.830961 np0005474935.novalocal sudo[146155]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Oct 07 17:24:50.190469 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.387s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.192212 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'target_lun': 0, 'volume_id': 'dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 'auth_method': 'CHAP', 'auth_username': 'nETx4SGpMWnUPc8CUjQp', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405a66b37664cc24a3fb66965941', 'path': '/dev/sda'}), 'kwargs': {'force': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:24:50.192735 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:24:50.193475 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:24:50.195192 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.210577 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.210842 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e953099d-0362-49ea-8089-ef46a0f1a018]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:24:50.211323 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:50.211751 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:24:50.211751 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:24:50.211751 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:24:50.212132 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:24:50.212913 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.227172 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.227527 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[12a1924f-aa37-4a28-9912-406c23023ad1]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.228545 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.247668 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.247946 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e6c8506-ef79-48f6-af7f-5266d79bd340]: (4, ('tcp: [28] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.248441 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 (non-flash) Oct 07 17:24:50.248441 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:24:50.249138 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [28] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 (non-flash) Oct 07 17:24:50.249138 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:24:50.250413 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9d7f54400>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1'): ({'sda'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:24:50.250876 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sda {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:24:50.251850 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.258079 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[939db472-4ab8-483c-b660-5f6cd3a37a33]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:24:50.260439 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:24:50.262272 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[939db472-4ab8-483c-b660-5f6cd3a37a33]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.263026 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sda {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:24:50.263864 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.270150 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.006s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.270476 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[49a0603e-78ab-4ec7-a1e3-c80227abc933]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.270880 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:24:50.271666 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.334323 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.062s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.334648 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fbbbba6-d89d-4c05-8dbf-8a297ca5e630]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.335198 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:24:50.335538 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sda have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:24:50.335993 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:24:50.336930 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.346554 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.346867 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6299eda-48aa-4057-8747-31ea3b7071bd]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.347271 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:50.348454 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.367537 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --logout" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.367790 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a5514fb-3006-4e35-b48e-407fff6e1d16]: (4, ('Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1, portal: 38.102.83.9,3260]\nLogout of [sid: 28, target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.368247 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1, portal: 38.102.83.9,3260] Oct 07 17:24:50.368247 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 28, target: iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1, portal: 38.102.83.9,3260] successful. Oct 07 17:24:50.368247 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:50.369197 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:24:50.378542 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dccf901c-7f89-4c9d-ad67-fb86447eb9d1 -p 38.102.83.9:3260 --op delete" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:24:50.378844 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b51d4ad1-a7b2-46c7-9151-14e040e0dafd]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:24:50.379320 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:24:50.379814 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.186s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:24:50.380158 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (187ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:24:53.907385 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.manager [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Encryption key ID for volume dccf901c-7f89-4c9d-ad67-fb86447eb9d1 already matches encryption key ID in backup 86183940-9c1b-440a-a9eb-6e047a93f398. {{(pid=104204) _run_restore /opt/stack/cinder/cinder/backup/manager.py:874}} Oct 07 17:24:53.929446 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-adba2054-dd2f-43eb-aab6-c83161e0e730 tempest-VolumesBackupsTest-2000989888 None] Finished restoring backup 86183940-9c1b-440a-a9eb-6e047a93f398 to volume dccf901c-7f89-4c9d-ad67-fb86447eb9d1. Oct 07 17:24:56.702880 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] Delete backup started, backup: 86183940-9c1b-440a-a9eb-6e047a93f398. Oct 07 17:24:56.705380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:24:56.705380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:24:56.705380 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] delete started, backup: 86183940-9c1b-440a-a9eb-6e047a93f398, container: tempest-volumesbackupstest-backup-container-1800894867, prefix: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:24:56.785987 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_metadata', 'volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:24:56.820167 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00001 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:56.854368 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00002 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:56.884856 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00003 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:56.916368 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00004 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:56.945400 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00005 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:56.981212 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00006 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.019767 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00007 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.055479 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00008 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.086061 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00009 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.116919 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00010 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.153442 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00011 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.182075 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00012 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.211603 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00013 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.242076 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00014 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.281072 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00015 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.306355 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00016 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.332287 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00017 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.360171 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00018 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.386485 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00019 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.412359 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00020 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.434635 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398-00021 in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.457441 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_metadata in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.482594 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_f9c09f88-a5c4-40ff-bfb9-267130e25ed2/20251007212416/az_nova_backup_86183940-9c1b-440a-a9eb-6e047a93f398_sha256file in container: tempest-volumesbackupstest-backup-container-1800894867. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:24:57.482994 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] delete 86183940-9c1b-440a-a9eb-6e047a93f398 finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:24:57.487810 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:24:57.505808 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] Created reservations ['45d1ec82-1d97-43d9-99c0-699cd484156a', 'fb289133-dc42-47f3-bc23-2452ad1e5635'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:24:57.531909 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-6c4efbbd-46cb-4e89-872a-c3cda8dc51f1 tempest-VolumesBackupsTest-2000989888 None] Delete backup finished, backup 86183940-9c1b-440a-a9eb-6e047a93f398 deleted. Oct 07 17:25:07.529827 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=63,cinder:UPDATE=13,cinder:INSERT=4 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:25:09.831175 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:25:09.831676 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:25:50.778976 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:25:52.786786 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Create backup started, backup: eafcb882-ce54-4490-ba4a-8b026583da64 volume: 325a0fe3-defe-4434-af3a-7f6788cdeea2. Oct 07 17:25:52.791040 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1206883695',created_at=2025-10-07T21:25:52Z,data_timestamp=2025-10-07T21:25:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1585407189',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=eafcb882-ce54-4490-ba4a-8b026583da64,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7124796c9a81406abe531b7e1755ce19',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='380e905f32ef429a8500f965eb48373f',volume_id=325a0fe3-defe-4434-af3a-7f6788cdeea2) Oct 07 17:25:54.420570 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:25:54.421281 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:25:54.421404 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:25:54.424180 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:54.432086 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:54.432822 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6b85345-4bd2-4248-8242-7bf9eae353e1]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[77b46889-1b85-47a4-8b5b-82328aca3a8c]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:54.436210 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:25:54.436789 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[77b46889-1b85-47a4-8b5b-82328aca3a8c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:54.437086 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:25:54.437587 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:25:54.440740 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:54.448413 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:54.449135 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbea3a8b-1ec1-4e45-a0d3-cbefb59621f0]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:54.450216 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:54.466188 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:54.467280 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[8e14f147-99a1-4bd4-a78a-ab6ba498a133]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:54.468005 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ec51351-6ca9-4bbe-8c3d-16dbd6f68b08]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:54.468744 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:54.485170 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:25:54.485816 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:25:54.488254 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:25:54.489621 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe1c:67e', 'fe80::dc67:a5ff:fe8f:a09f'] Oct 07 17:25:54.490004 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:25:54.490438 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:25:54.491082 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (69ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe1c:67e', 'fe80::dc67:a5ff:fe8f:a09f']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:25:58.503476 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:25:58.504667 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'target_lun': 0, 'volume_id': 'dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'auth_method': 'CHAP', 'auth_username': 'gHzhjHtU3CKL5s4T7NYQ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:25:58.504966 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:25:58.505822 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:25:58.506090 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'target_lun': 0, 'volume_id': 'dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'auth_method': 'CHAP', 'auth_username': 'gHzhjHtU3CKL5s4T7NYQ', 'auth_password': 'wtwX38hraoXLSvqp', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:25:58.507630 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:58.516797 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[9350a752-b87a-4070-ab0f-ba51df156629]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:25:58.518870 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:25:58.520599 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9350a752-b87a-4070-ab0f-ba51df156629]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:58.521519 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:25:58.522213 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:25:58.522617 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:25:58.523921 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:58.539045 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:58.539303 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[efcea9f8-c7ba-46fa-a0b1-9cd28a3cbccf]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:58.539793 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:25:58.539793 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:58.540857 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:58.557705 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:58.557968 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1ee7b80-3fd7-4cfc-b764-afc1b2b13bf8]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:58.558497 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d] added Oct 07 17:25:58.558497 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:25:58.558497 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:25:58.558497 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:25:58.558497 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:58.558753 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:25:58.559393 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.037(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:25:58.559726 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:25:59.561278 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:25:59.562556 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.578285 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.578665 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e7acb3d-8a21-4bd8-bbf0-d3de9e15bc20]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:25:59.580475 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.583928 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.598354 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.598731 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[31b76184-f109-4629-8d7c-62d8403fd239]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.599245 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.600312 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.617167 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.617534 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e240a02-7247-4d8b-9886-7328e61350ba]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.618144 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.619272 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.auth.username -v gHzhjHtU3CKL5s4T7NYQ {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.636503 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.auth.username -v gHzhjHtU3CKL5s4T7NYQ" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.636796 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7766a97-22fe-44fa-be0f-64f491a7a943]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.637288 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'gHzhjHtU3CKL5s4T7NYQ'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.638567 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.655735 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.656002 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[0111072e-96bc-433d-b836-a2f7f7b5e1fe]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.656679 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.657662 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.671475 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.671749 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[509983d5-7508-4e0e-b142-bf1e872123ce]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.672216 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:25:59.672216 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:25:59.672511 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:25:59.672511 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:25:59.673607 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.699723 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --login" returned: 0 in 0.026s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.700118 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[61212135-fd23-4736-9e37-1cc7b9156b3b]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.700933 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d, portal: 38.102.83.9,3260] successful. Oct 07 17:25:59.700933 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.703373 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.719776 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.720008 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[15cf09f1-76da-4d4b-99bb-5e60d48ef574]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.720543 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:25:59.721604 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.740993 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.741271 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0f71deb-cdd4-4d7e-a962-6fdc7a1fb2d7]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\ntcp: [30] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.741807 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:25:59.741807 np0005474935.novalocal cinder-backup[104204]: tcp: [30] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d (non-flash) Oct 07 17:25:59.741807 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:25:59.742063 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:25:59.742063 np0005474935.novalocal cinder-backup[104204]: tcp: [30] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d (non-flash) Oct 07 17:25:59.742063 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:25:59.742910 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.221s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:25:59.743173 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:25:59.744204 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] HCTL ('3', '-', '-', 0) found on session 30 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:25:59.744521 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:25:59.745616 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:25:59.806522 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.034s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:25:59.806522 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[8cf7accb-f46a-4d8d-9f51-85da680f6b80]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:25:59.806522 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 30 and hctl ['3', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:26:00.784961 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 30 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:26:00.785772 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'target_lun': 0, 'volume_id': 'dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'auth_method': 'CHAP', 'auth_username': 'gHzhjHtU3CKL5s4T7NYQ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:26:00.786645 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.281s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:00.787009 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2282ms) {'type': 'block', 'scsi_wwn': '3600140525f3e0e0ec0f4ae7a55229a9f', 'path': '/dev/sdb'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:00.787633 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:00.816368 np0005474935.novalocal sudo[146755]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Oct 07 17:26:01.232522 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.445s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:01.237525 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:26:01.248075 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1206883695,backup: eafcb882-ce54-4490-ba4a-8b026583da64. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:26:01.324709 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] generate_object_name_prefix: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64 {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:26:01.331747 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] starting backup of volume: 325a0fe3-defe-4434-af3a-7f6788cdeea2, volume size: 1073741824, object names prefix volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:26:01.612485 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:01.731367 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:01.731587 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:01.890723 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:01.891472 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:02.352070 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:02.485610 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:02.485610 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:02.597648 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:02.597918 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:03.621020 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:03.741507 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:03.741691 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:03.918605 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:03.918804 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:04.424852 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:04.638528 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:04.639241 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:04.822675 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:04.822865 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:05.262748 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:05.396634 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:05.396892 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:05.552173 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:05.552448 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:05.739138 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:05.865483 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:05.865751 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:05.993250 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:05.993413 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:06.467864 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:06.573428 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:06.573694 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:06.724260 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:06.724420 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:06.986937 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:07.102842 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:07.103109 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:07.213799 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:07.214008 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:08.022319 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:08.159053 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:08.159311 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:08.340441 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:08.340704 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:08.781089 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:08.917640 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:08.917858 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:09.041204 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:09.041961 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:09.274865 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:09.407161 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:09.407458 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:09.555625 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:09.555625 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:09.832223 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:26:09.832404 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:26:10.034179 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:10.168365 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:10.169035 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:10.320611 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:10.320852 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:10.840680 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:10.976395 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:10.976572 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:11.141966 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:11.142238 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:11.798447 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:11.961517 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:11.961618 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:12.187786 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:12.188051 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:12.623663 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:12.757953 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:12.758263 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:12.911833 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:12.912435 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:13.400555 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:13.545301 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:13.545301 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:13.698918 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:13.699225 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:13.982134 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:14.213234 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:14.213556 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:14.370525 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:14.370799 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:14.844149 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:14.984622 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:14.985495 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:15.118616 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:15.118843 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:15.402092 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:15.536656 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:15.536909 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:15.692411 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:15.692665 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:15.946899 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:16.083828 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:16.084834 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:16.235752 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:16.235920 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:16.590108 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Backing up chunk of data from volume. {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Oct 07 17:26:16.652322 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=104204) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Oct 07 17:26:16.652445 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] About to put_object {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Oct 07 17:26:16.755712 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup MD5 for volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Oct 07 17:26:16.755980 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Calling eventlet.sleep(0) {{(pid=104204) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Oct 07 17:26:16.762859 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:26:16.775714 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:26:16.776064 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:26:16.782921 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:26:16.783219 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:26:16.788361 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:26:16.788859 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1206883695, sha256file filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:26:16.883436 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:26:16.884315 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1206883695, metadata filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:26:16.914513 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "eafcb882-ce54-4490-ba4a-8b026583da64",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1585407189",\n "created_at": "2025-10-07 21:25:52+00:00",\n "objects": [\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "325a0fe3-defe-4434-af3a-7f6788cdeea2",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"325a0fe3-defe-4434-af3a-7f6788cdeea2\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"380e905f32ef429a8500f965eb48373f\\", \\"project_id\\": \\"7124796c9a81406abe531b7e1755ce19\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:25:00.000000\\", \\"launched_at\\": \\"2025-10-07T21:25:00.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-392259792\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 0\\", \\"provider_auth\\": \\"CHAP VHUdYASNQ3M3U9EyPA2L yi5oShyjyZFgG5AE\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:25:00.000000\\", \\"updated_at\\": \\"2025-10-07T21:25:52.000000\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:26:16.919488 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] backup eafcb882-ce54-4490-ba4a-8b026583da64 finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:26:17.084130 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.122172 np0005474935.novalocal sudo[146783]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Oct 07 17:26:17.543100 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.459s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.545018 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'target_lun': 0, 'volume_id': 'dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 'auth_method': 'CHAP', 'auth_username': 'gHzhjHtU3CKL5s4T7NYQ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '3600140525f3e0e0ec0f4ae7a55229a9f', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:26:17.545413 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:26:17.547404 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:26:17.548725 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.569507 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.569825 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[856a53d9-2978-449f-b3cc-831aad7259c9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:26:17.570401 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:17.570821 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:26:17.570821 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:26:17.570821 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:26:17.571225 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:26:17.572253 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.588965 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.589272 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9488b6f3-dcff-4df8-b5ee-5341d5a8f9d4]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2\n38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.590491 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.608771 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.609093 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[10a1f775-8186-4362-bfbf-13b8dc9bfee0]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\ntcp: [30] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.609676 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:17.609676 np0005474935.novalocal cinder-backup[104204]: tcp: [30] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d (non-flash) Oct 07 17:26:17.609676 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:17.609987 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:17.609987 np0005474935.novalocal cinder-backup[104204]: tcp: [30] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d (non-flash) Oct 07 17:26:17.609987 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:26:17.611390 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9d83440e0>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d'): ({'sdb'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:26:17.611873 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sdb {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:26:17.612873 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.622664 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[db9b6271-d6e7-47f8-a3d4-4b263c49eb99]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:26:17.623608 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:26:17.625537 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[db9b6271-d6e7-47f8-a3d4-4b263c49eb99]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.626420 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sdb {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:26:17.627609 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.638244 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.638550 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8d93dcb-53fe-4508-83fc-94c2150f52e8]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.639016 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:26:17.639949 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.713409 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.073s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.713808 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8e38d4b-ff9a-4c2b-84b4-892e6204f080]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.714450 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:26:17.714814 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:26:17.715658 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:26:17.716772 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.737497 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.737850 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[90a722d1-767b-407b-9a20-6974c36858f4]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.738391 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:17.739569 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.769824 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --logout" returned: 0 in 0.028s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.770133 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[97ddc205-4e25-4136-8b0e-361ca49771db]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d, portal: 38.102.83.9,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.770732 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d, portal: 38.102.83.9,3260] Oct 07 17:26:17.770732 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d, portal: 38.102.83.9,3260] successful. Oct 07 17:26:17.770732 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:17.771833 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:17.788497 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-dff06f9a-0d9c-47ce-a1d9-6cd86d82998d -p 38.102.83.9:3260 --op delete" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:17.788645 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8b01a69-d081-4e5e-bf71-2da1217cd269]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:17.789100 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:17.789378 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.243s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:17.789599 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (243ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:21.339195 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3ea1f0fb-5d80-447e-a927-5dd9cb010369 tempest-VolumesBackupsTest-2000989888 None] Create backup finished. backup: eafcb882-ce54-4490-ba4a-8b026583da64. Oct 07 17:26:21.857281 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Create backup started, backup: e55489cf-b8db-4e8e-8d63-3ec45d270e6c volume: 325a0fe3-defe-4434-af3a-7f6788cdeea2. Oct 07 17:26:21.861420 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container=None,created_at=2025-10-07T21:26:21Z,data_timestamp=2025-10-07T21:26:21Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=e55489cf-b8db-4e8e-8d63-3ec45d270e6c,metadata={},num_dependent_backups=0,object_count=0,parent=Backup(eafcb882-ce54-4490-ba4a-8b026583da64),parent_id='eafcb882-ce54-4490-ba4a-8b026583da64',project_id='7124796c9a81406abe531b7e1755ce19',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='380e905f32ef429a8500f965eb48373f',volume_id=325a0fe3-defe-4434-af3a-7f6788cdeea2) Oct 07 17:26:23.422556 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:26:23.423322 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:26:23.423322 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:26:23.425604 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:23.433094 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:23.433273 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[389350cc-63b6-4cbb-af3b-2f4212e64d0d]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[1d619158-f521-4256-9797-0197001354e2]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:23.434810 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:26:23.435525 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d619158-f521-4256-9797-0197001354e2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:23.436076 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:26:23.436369 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:26:23.438591 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:23.446917 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:23.447492 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[97a63474-5214-4c71-80a9-c8037fc281bd]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:23.448570 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:23.458445 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:23.459008 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f5a1e5a-3a4c-4386-9c6d-1cd0cb7b54a5]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:23.459960 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1de96d7a-02b2-438f-b561-73603da1dfcf]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:23.460286 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:23.477613 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:26:23.477874 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:26:23.479623 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:26:23.480411 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe1c:67e', 'fe80::dc67:a5ff:fe8f:a09f'] Oct 07 17:26:23.480576 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:26:23.480737 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:26:23.481071 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (58ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe1c:67e', 'fe80::dc67:a5ff:fe8f:a09f']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:27.488616 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:26:27.490175 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642', 'target_lun': 0, 'volume_id': '99abab3e-03e1-4abc-85e0-a773b6e48642', 'auth_method': 'CHAP', 'auth_username': 'q489eLNh3M3WZvLjK5SW', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:26:27.490617 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:26:27.491440 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:26:27.491814 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642', 'target_lun': 0, 'volume_id': '99abab3e-03e1-4abc-85e0-a773b6e48642', 'auth_method': 'CHAP', 'auth_username': 'q489eLNh3M3WZvLjK5SW', 'auth_password': 'yLuzDi7tg5i89yhe', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:26:27.495812 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:27.502863 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[05bc1ed7-fbb8-4d32-8af5-cad68d624377]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:26:27.505788 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:26:27.508413 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[05bc1ed7-fbb8-4d32-8af5-cad68d624377]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:27.509589 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:26:27.510297 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:26:27.510788 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:26:27.512137 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:27.527607 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260" returned: 21 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:27.528017 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab563667-c1a7-4524-bcd8-1bfab8096da0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:27.528544 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:26:27.528544 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:27.529595 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:27.546953 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:27.547463 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcd5a72d-346e-4f9f-8e73-3b2649baa447]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:27.547994 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642] added Oct 07 17:26:27.547994 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:26:27.547994 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:26:27.547994 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:26:27.547994 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:27.548381 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:26:27.549057 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.038(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:26:27.549500 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:26:28.551797 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:26:28.553923 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.568773 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.569231 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[72a03c48-83b6-4d02-8cf6-da66a0fca012]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:26:28.571010 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.574182 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.586653 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.586981 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b152269f-5754-495b-8938-00040abf532f]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.587619 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.588713 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.603542 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.603791 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[65b2ba1d-507e-4bec-8091-8e275f02c07b]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.604235 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.605305 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v q489eLNh3M3WZvLjK5SW {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.621743 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.auth.username -v q489eLNh3M3WZvLjK5SW" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.622099 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e2ab9ae-9261-43c1-9765-45919fb6c38b]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.622653 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'q489eLNh3M3WZvLjK5SW'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.623897 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.640193 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.640526 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[54ab1763-8719-4b63-91c5-3bdb93cea8f3]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.641079 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.642102 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.659146 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.659519 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3378aaf9-cd97-4bcf-90f7-7d2796ff0448]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.659946 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:28.659946 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:28.660212 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:28.660212 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:26:28.661351 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.688641 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --login" returned: 0 in 0.027s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.688919 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4076104-9cee-4b1b-9b51-5a544b67039f]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.689461 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642, portal: 38.102.83.9,3260] successful. Oct 07 17:26:28.689461 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.691954 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.709669 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.709950 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ebb495a-ec42-45d2-a01d-1f1ebb1f8385]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.710467 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:28.711618 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.730031 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.730387 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[75f79cfd-3171-4b52-9f95-b200187df76f]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\ntcp: [31] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.730813 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:28.730813 np0005474935.novalocal cinder-backup[104204]: tcp: [31] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 (non-flash) Oct 07 17:26:28.730813 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:28.731097 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:28.731097 np0005474935.novalocal cinder-backup[104204]: tcp: [31] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 (non-flash) Oct 07 17:26:28.731097 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:26:28.731618 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.221s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:28.731885 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:26:28.732856 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] HCTL ('3', '-', '-', 0) found on session 31 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:26:28.733121 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:26:28.734221 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:28.809642 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.031s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:28.809642 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2adfdf85-73e4-4bf0-ada0-e7020c8a07bd]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:28.809642 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 31 and hctl ['3', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:26:29.768360 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 31 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:26:29.769577 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642', 'target_lun': 0, 'volume_id': '99abab3e-03e1-4abc-85e0-a773b6e48642', 'auth_method': 'CHAP', 'auth_username': 'q489eLNh3M3WZvLjK5SW', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:26:29.770025 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.279s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:29.770370 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2279ms) {'type': 'block', 'scsi_wwn': '360014052e0db96343584091aa2a949b7', 'path': '/dev/sdb'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:29.770929 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:29.820778 np0005474935.novalocal sudo[147032]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Oct 07 17:26:30.302274 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.531s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:30.308855 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:26:30.327217 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _read_sha256file started, container name: tempest-volumesbackupstest-backup-container-1206883695, sha256 filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_sha256file. {{(pid=104204) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:336}} Oct 07 17:26:30.380140 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _read_sha256file finished. {{(pid=104204) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:343}} Oct 07 17:26:30.394468 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _create_container started, container: volumebackups,backup: e55489cf-b8db-4e8e-8d63-3ec45d270e6c. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:26:30.475430 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] generate_object_name_prefix: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:26:30.482317 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] starting backup of volume: 325a0fe3-defe-4434-af3a-7f6788cdeea2, volume size: 1073741824, object names prefix volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:26:39.540229 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:26:39.552860 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:26:39.553160 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:26:39.558174 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:26:39.558497 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:26:39.563470 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:26:39.563951 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file started, container name: volumebackups, sha256file filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:26:39.682292 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:26:39.683674 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _write_metadata started, container name: volumebackups, metadata filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:26:39.719053 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "e55489cf-b8db-4e8e-8d63-3ec45d270e6c",\n "backup_name": null,\n "created_at": "2025-10-07 21:26:21+00:00",\n "objects": [],\n "parent_id": "eafcb882-ce54-4490-ba4a-8b026583da64",\n "version": "1.0.0",\n "volume_id": "325a0fe3-defe-4434-af3a-7f6788cdeea2",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"325a0fe3-defe-4434-af3a-7f6788cdeea2\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"380e905f32ef429a8500f965eb48373f\\", \\"project_id\\": \\"7124796c9a81406abe531b7e1755ce19\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:25:00.000000\\", \\"launched_at\\": \\"2025-10-07T21:25:00.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-392259792\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 0\\", \\"provider_auth\\": \\"CHAP VHUdYASNQ3M3U9EyPA2L yi5oShyjyZFgG5AE\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:25:00.000000\\", \\"updated_at\\": \\"2025-10-07T21:26:21.000000\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:26:39.724045 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] backup e55489cf-b8db-4e8e-8d63-3ec45d270e6c finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:26:39.902115 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:39.928050 np0005474935.novalocal sudo[147052]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Oct 07 17:26:40.323134 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.421s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.324248 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642', 'target_lun': 0, 'volume_id': '99abab3e-03e1-4abc-85e0-a773b6e48642', 'auth_method': 'CHAP', 'auth_username': 'q489eLNh3M3WZvLjK5SW', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014052e0db96343584091aa2a949b7', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:26:40.324547 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:26:40.324972 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:26:40.326535 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.343896 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.344394 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e76c4270-2a25-49ff-940b-0c63463a548d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:26:40.344801 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:40.345258 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:26:40.345258 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:26:40.345258 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:26:40.345656 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:26:40.346633 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.360153 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.360485 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[26ffa11a-8c63-4ad7-897e-c7fce59665fd]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2\n38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.361640 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.376235 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.376770 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f5af187-6a3c-4b20-9f01-0399fe372b5a]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\ntcp: [31] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.377153 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:40.377153 np0005474935.novalocal cinder-backup[104204]: tcp: [31] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 (non-flash) Oct 07 17:26:40.377153 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:40.377495 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:40.377495 np0005474935.novalocal cinder-backup[104204]: tcp: [31] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 (non-flash) Oct 07 17:26:40.377495 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:26:40.378677 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9f645d260>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642'): ({'sdb'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:26:40.379105 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sdb {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:26:40.380123 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.388738 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[61d39df1-4f13-49d1-b9a7-4a4f09d06fba]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:26:40.391036 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:26:40.393189 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[61d39df1-4f13-49d1-b9a7-4a4f09d06fba]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.393803 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sdb {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:26:40.394807 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.404024 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.404388 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8211a64-8fd8-4235-bc5c-76dc74a4979b]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.404791 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:26:40.405878 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.459403 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.053s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.459704 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[592b59c8-412a-4036-91f5-45ba3bd2e3f7]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.460171 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:26:40.460509 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:26:40.461192 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:26:40.463102 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.472278 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.009s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.473685 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[28abf55c-aec2-4fcd-9c17-b655d19edce8]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.474038 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:40.474762 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.496314 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --logout" returned: 0 in 0.021s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.496630 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[47e401e6-36b7-4658-bf65-6e846ba7cba6]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642, portal: 38.102.83.9,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.497214 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642, portal: 38.102.83.9,3260] Oct 07 17:26:40.497214 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642, portal: 38.102.83.9,3260] successful. Oct 07 17:26:40.497214 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:40.498198 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:40.514811 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-99abab3e-03e1-4abc-85e0-a773b6e48642 -p 38.102.83.9:3260 --op delete" returned: 0 in 0.016s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:40.515054 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a495c26-3c2a-46ae-bc15-f006a22588a6]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:40.515561 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:40.516039 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.191s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:40.516408 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (191ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:44.079946 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-fbd36ee6-49bd-41d1-bb36-cfd9177daf02 tempest-VolumesBackupsTest-2000989888 None] Create backup finished. backup: e55489cf-b8db-4e8e-8d63-3ec45d270e6c. Oct 07 17:26:44.637210 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] Delete backup started, backup: e55489cf-b8db-4e8e-8d63-3ec45d270e6c. Oct 07 17:26:44.638238 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:26:44.638482 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:26:44.638685 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] delete started, backup: e55489cf-b8db-4e8e-8d63-3ec45d270e6c, container: volumebackups, prefix: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:26:44.706014 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c_metadata', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:26:44.742890 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c_metadata in container: volumebackups. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:26:44.771829 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212630/az_nova_backup_e55489cf-b8db-4e8e-8d63-3ec45d270e6c_sha256file in container: volumebackups. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:26:44.772170 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] delete e55489cf-b8db-4e8e-8d63-3ec45d270e6c finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:26:44.776731 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:26:44.804180 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] Created reservations ['f6de791c-86a1-47b4-8293-b2f8a6ec6b3f', '95f1385e-a4f3-4ae6-b0eb-d1e75be9fe40'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:26:44.834286 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-4c6ce9eb-3231-4711-855d-7db65707af1b tempest-VolumesBackupsTest-2000989888 None] Delete backup finished, backup e55489cf-b8db-4e8e-8d63-3ec45d270e6c deleted. Oct 07 17:26:45.806820 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Create backup started, backup: 6f570b01-4615-4d3b-a9a4-3edc228cecc2 volume: 325a0fe3-defe-4434-af3a-7f6788cdeea2. Oct 07 17:26:45.811356 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1485026413',created_at=2025-10-07T21:26:45Z,data_timestamp=2025-10-07T21:26:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1133289698',encryption_key_id=None,fail_reason=None,host='np0005474935.novalocal',id=6f570b01-4615-4d3b-a9a4-3edc228cecc2,metadata={},num_dependent_backups=0,object_count=0,parent=Backup(eafcb882-ce54-4490-ba4a-8b026583da64),parent_id='eafcb882-ce54-4490-ba4a-8b026583da64',project_id='7124796c9a81406abe531b7e1755ce19',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='380e905f32ef429a8500f965eb48373f',volume_id=325a0fe3-defe-4434-af3a-7f6788cdeea2) Oct 07 17:26:47.395249 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:26:47.396036 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:26:47.396036 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.9', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:26:47.397611 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:47.409405 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:47.409700 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f428ef9-7275-4de3-a2ff-70ec8133f9a0]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1ec65087f188\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[5307fc0a-46c2-40b5-86a7-e77aa1bdd646]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: with open_scini_device() as fd: Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: return next(self.gen) Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^ Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:47.412552 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Oct 07 17:26:47.413271 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5307fc0a-46c2-40b5-86a7-e77aa1bdd646]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:47.413921 np0005474935.novalocal cinder-backup[104204]: ERROR os_brick.initiator.connectors.scaleio [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Oct 07 17:26:47.414422 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.scaleio [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Oct 07 17:26:47.415549 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:47.432835 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:47.433048 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3dd44dbb-4cdf-48ca-9795-3f975256b2fe]: (4, ('/dev/vda2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:47.434304 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:47.452654 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:47.452990 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e1df620-cd0f-4b4d-8e60-3dc6f0131b51]: (4, ('3a138e04-7996-4722-a0f3-d2f41a6c276a\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:47.454375 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[eff14ca6-8c14-44a4-a3ab-02ef2bb8c293]: (4, 'c422b812-0c47-4194-b485-cb9fe1c5f3ea') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:47.454963 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): nvme version {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:47.473518 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] 'nvme version' failed. Not Retrying. {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:26:47.473866 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] nvme not present on system {{(pid=104204) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Oct 07 17:26:47.476482 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=104204) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Oct 07 17:26:47.477745 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.lightos [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 and IP(s) are ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe1c:67e', 'fe80::dc67:a5ff:fe8f:a09f'] Oct 07 17:26:47.478022 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Oct 07 17:26:47.478262 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.lightos [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701 dsc: {{(pid=104204) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Oct 07 17:26:47.478820 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] <== get_connector_properties: return (81ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.9', 'host': 'np0005474935.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1ec65087f188', 'do_local_attach': False, 'uuid': '3a138e04-7996-4722-a0f3-d2f41a6c276a', 'system uuid': 'c422b812-0c47-4194-b485-cb9fe1c5f3ea', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b14c95dd-1320-4fa1-b8bd-406067ae4701', 'found_dsc': '', 'host_ips': ['38.102.83.9', 'fe80::f816:3eff:feac:8db4', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::d028:1bff:fe66:5a4b', 'fe80::fc16:3eff:fe1c:67e', 'fe80::dc67:a5ff:fe8f:a09f']} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:51.491809 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connector [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Factory for iscsi on None {{(pid=104204) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Oct 07 17:26:51.493036 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a', 'target_lun': 0, 'volume_id': '127799f1-efe0-4649-baa6-c2d5e2634f2a', 'auth_method': 'CHAP', 'auth_username': 'NTYQ86JwfXdmTEQ5W54w', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:26:51.493349 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:26:51.494143 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:26:51.494453 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a', 'target_lun': 0, 'volume_id': '127799f1-efe0-4649-baa6-c2d5e2634f2a', 'auth_method': 'CHAP', 'auth_username': 'NTYQ86JwfXdmTEQ5W54w', 'auth_password': 'fB5eLsGxnZye6N9P', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=104204) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Oct 07 17:26:51.495960 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:51.507663 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[05c8cfd7-a6fc-4789-9344-249d1e036483]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:26:51.509106 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:26:51.510210 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[05c8cfd7-a6fc-4789-9344-249d1e036483]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:51.511139 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:26:51.511888 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:26:51.512235 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:26:51.513241 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:51.530616 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260" returned: 21 in 0.017s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:51.530929 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[52656396-acc1-4d13-83f0-0be6ce010d90]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:51.531372 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Oct 07 17:26:51.531372 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:51.532312 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --interface default --op new {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:51.551227 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:51.551531 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[4548521a-04b7-4c8e-961a-664e06f7de26]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:51.552053 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.9,3260,-1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a] added Oct 07 17:26:51.552053 np0005474935.novalocal cinder-backup[104204]: stderr=iscsiadm: Config file line 381 does not have value Oct 07 17:26:51.552053 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:26:51.552053 np0005474935.novalocal cinder-backup[104204]: iscsiadm: Config file line 381 does not have value Oct 07 17:26:51.552053 np0005474935.novalocal cinder-backup[104204]: {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:51.552494 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Retrying to connect to iSCSI portal 38.102.83.9:3260 {{(pid=104204) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Oct 07 17:26:51.553174 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.041(s), this was the 1st time calling it. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Oct 07 17:26:51.553567 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.utils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=104204) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Oct 07 17:26:52.555482 np0005474935.novalocal cinder-backup[104204]: INFO os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Trying to connect to iSCSI portal 38.102.83.9:3260 Oct 07 17:26:52.557112 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.575600 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.575922 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[5daddf88-0feb-4be8-8640-3f963932f390]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.9\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.name = iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.tpgt = -1 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.startup = automatic Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.leading_login = No Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.iscsi_ifacename = default Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.net_ifacename = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.ipaddress = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.prefix_len = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.hwaddress = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.transport_name = tcp Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.initiatorname = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.vlan_id = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.vlan_priority = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.vlan_state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.iface_num = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.mtu = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.port = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.bootproto = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.subnet_mask = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.gateway = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id_state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_alt_client_id = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_dns = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_learn_iqn = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_req_vendor_id_state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id_state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_vendor_id = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.dhcp_slp_da = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.fragmentation = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.gratuitous_arp = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.incoming_forwarding = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tos_state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tos = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.ttl = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.delayed_ack = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tcp_nagle = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf_state = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tcp_wsf = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timer_scale = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.tcp_timestamp = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.redirect = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.def_task_mgmt_timeout = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.header_digest = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.data_digest = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.immediate_data = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.initial_r2t = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.data_seq_inorder = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.data_pdu_inorder = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.erl = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.max_receive_data_len = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.first_burst_len = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.max_outstanding_r2t = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.max_burst_len = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.chap_auth = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.bidi_chap = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.strict_login_compliance = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.discovery_auth = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: iface.discovery_logout = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.discovery_address = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.discovery_port = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.discovery_type = fw Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.initial_cmdsn = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.initial_login_retry_max = 8 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.xmit_thread_priority = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.cmds_max = 128 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.queue_depth = 32 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.nr_sessions = 1 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.auth.authmethod = None Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password = *** Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.auth.username_in = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.auth.password_in = Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.auth.chap_algs = SHA3-256,SHA256 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.timeo.replacement_timeout = 120 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.abort_timeout = 15 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.lu_reset_timeout = 30 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.tgt_reset_timeout = 30 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.err_timeo.host_reset_timeout = 60 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FastAbort = Yes Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.InitialR2T = No Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ImmediateData = Yes Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.FirstBurstLength = 262144 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxBurstLength = 16776192 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Retain = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.DefaultTime2Wait = 2 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxConnections = 1 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.MaxOutstandingR2T = 1 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.iscsi.ERL = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.scan = auto Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.reopen_max = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.session.conn_reopen_log_freq = 1 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].address = 38.102.83.9 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].port = 3260 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].startup = manual Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.window_size = 524288 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].tcp.type_of_service = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.logout_timeout = 15 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.login_timeout = 15 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.auth_timeout = 45 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_interval = 5 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].timeo.noop_out_timeout = 5 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.HeaderDigest = None Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.DataDigest = None Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.IFMarker = No Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: node.conn[0].iscsi.OFMarker = No Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: # END RECORD Oct 07 17:26:52.577864 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.581015 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.scan -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.601519 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.602007 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd9b6143-a237-46a5-8dec-ee96b19138b7]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.602432 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.603606 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.625762 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.626005 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[6dbfa1b1-e961-4f94-a71c-5510efde7b70]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.626463 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.627524 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.auth.username -v NTYQ86JwfXdmTEQ5W54w {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.647008 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.auth.username -v NTYQ86JwfXdmTEQ5W54w" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.647288 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[4925f21a-88f8-4f2b-8217-836fd76f2f07]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.647899 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NTYQ86JwfXdmTEQ5W54w'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.649178 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.auth.password -v *** {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.668600 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.668856 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[504db45b-1e6c-4f6a-b3b3-b485f5efd1c1]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.669587 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.670564 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.693861 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.694205 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[acbface4-7483-46be-a254-edcf49e5db1f]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.694664 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:52.694664 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:52.694921 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:52.694921 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:26:52.695937 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --login {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.730575 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --login" returned: 0 in 0.033s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.733670 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[3762f7b8-af29-48e4-98ac-db4307e95608]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.734315 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a, portal: 38.102.83.9,3260] successful. Oct 07 17:26:52.734315 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.735739 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.startup -v automatic {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.750840 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.startup -v automatic" returned: 0 in 0.015s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.751152 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dd9da67-ecde-43e2-9288-e703f5b74f3c]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.751713 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:26:52.752854 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.767021 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.014s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.767265 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[b81e0900-72e2-40ca-9d68-3b8b65c60b24]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\ntcp: [32] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.767730 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:52.767730 np0005474935.novalocal cinder-backup[104204]: tcp: [32] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a (non-flash) Oct 07 17:26:52.767730 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:26:52.767977 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:26:52.767977 np0005474935.novalocal cinder-backup[104204]: tcp: [32] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a (non-flash) Oct 07 17:26:52.767977 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:26:52.768615 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_to_iscsi_portal-38.102.83.9:3260-iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.257s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:52.768884 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Connected to 38.102.83.9:3260 {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Oct 07 17:26:52.770013 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] HCTL ('3', '-', '-', 0) found on session 32 with lun 0 {{(pid=104204) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Oct 07 17:26:52.770272 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=104204) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Oct 07 17:26:52.771399 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:52.832106 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.022s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:52.832106 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d518437-8ab8-491c-928a-4f3aae57a69d]: (4, ('- - 0', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:26:52.832106 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 32 and hctl ['3', '*', '*', 0] yield: None {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:26:53.797150 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Searching for a device in session 32 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=104204) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Oct 07 17:26:53.798019 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a', 'target_lun': 0, 'volume_id': '127799f1-efe0-4649-baa6-c2d5e2634f2a', 'auth_method': 'CHAP', 'auth_username': 'NTYQ86JwfXdmTEQ5W54w', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=104204) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Oct 07 17:26:53.798898 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.305s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:26:53.799178 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] <== connect_volume: return (2305ms) {'type': 'block', 'scsi_wwn': '360014052968d53b94f044749f7485b49', 'path': '/dev/sdb'} {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:26:53.799823 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:26:53.826872 np0005474935.novalocal sudo[147288]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Oct 07 17:26:54.177259 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.377s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:26:54.184436 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:26:54.186404 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=83,cinder:UPDATE=22,cinder:INSERT=2 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:26:54.193645 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _read_sha256file started, container name: tempest-volumesbackupstest-backup-container-1206883695, sha256 filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_sha256file. {{(pid=104204) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:336}} Oct 07 17:26:54.246514 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _read_sha256file finished. {{(pid=104204) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:343}} Oct 07 17:26:54.258593 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1485026413,backup: 6f570b01-4615-4d3b-a9a4-3edc228cecc2. {{(pid=104204) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Oct 07 17:26:54.338846 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] generate_object_name_prefix: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2 {{(pid=104204) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Oct 07 17:26:54.345505 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] starting backup of volume: 325a0fe3-defe-4434-af3a-7f6788cdeea2, volume size: 1073741824, object names prefix volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2, availability zone: nova {{(pid=104204) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Oct 07 17:27:03.159232 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Oct 07 17:27:03.170824 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=104204) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Oct 07 17:27:03.171118 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-metadata' {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Oct 07 17:27:03.175987 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-metadata' available {{(pid=104204) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Oct 07 17:27:03.176264 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Getting metadata type 'volume-glance-metadata' {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Oct 07 17:27:03.180759 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.driver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] No metadata type 'volume-glance-metadata' available {{(pid=104204) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Oct 07 17:27:03.181209 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1485026413, sha256file filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2_sha256file. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Oct 07 17:27:03.300324 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _write_sha256file finished. {{(pid=104204) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Oct 07 17:27:03.301794 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1485026413, metadata filename: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2_metadata. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Oct 07 17:27:03.340102 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "6f570b01-4615-4d3b-a9a4-3edc228cecc2",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1133289698",\n "created_at": "2025-10-07 21:26:45+00:00",\n "objects": [],\n "parent_id": "eafcb882-ce54-4490-ba4a-8b026583da64",\n "version": "1.0.0",\n "volume_id": "325a0fe3-defe-4434-af3a-7f6788cdeea2",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"325a0fe3-defe-4434-af3a-7f6788cdeea2\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"380e905f32ef429a8500f965eb48373f\\", \\"project_id\\": \\"7124796c9a81406abe531b7e1755ce19\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005474935.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-10-07T21:25:00.000000\\", \\"launched_at\\": \\"2025-10-07T21:25:00.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-392259792\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.9:3260,iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 0\\", \\"provider_auth\\": \\"CHAP VHUdYASNQ3M3U9EyPA2L yi5oShyjyZFgG5AE\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"cb5faf91-e3d4-47d3-abe9-b252347dc403\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"5541d238-1514-4f23-ae57-3e747aca38c5\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-10-07T21:25:00.000000\\", \\"updated_at\\": \\"2025-10-07T21:26:45.000000\\"}}"\n}'. {{(pid=104204) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Oct 07 17:27:03.347738 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] backup 6f570b01-4615-4d3b-a9a4-3edc228cecc2 finished. {{(pid=104204) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Oct 07 17:27:03.531472 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:03.567433 np0005474935.novalocal sudo[147311]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Oct 07 17:27:03.876901 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.345s {{(pid=104204) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:03.878482 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.9:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a', 'target_lun': 0, 'volume_id': '127799f1-efe0-4649-baa6-c2d5e2634f2a', 'auth_method': 'CHAP', 'auth_username': 'NTYQ86JwfXdmTEQ5W54w', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014052968d53b94f044749f7485b49', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Oct 07 17:27:03.878707 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Oct 07 17:27:03.879354 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Oct 07 17:27:03.881105 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:03.891779 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.011s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:03.892124 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ee70d1c-79a2-45bc-9889-404108df0287]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a\n\tPortal: 38.102.83.9:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: iSNS: Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: STATIC: Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: Target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: Portal: 38.102.83.9:3260,-1 Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: Iface Name: default Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: FIRMWARE: Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: No targets found. Oct 07 17:27:03.892456 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:27:03.892643 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Regex to get portals from discoverydb: ^SENDTARGETS: Oct 07 17:27:03.892643 np0005474935.novalocal cinder-backup[104204]: .*?^DiscoveryAddress: 38.102.83.9,3260.*? Oct 07 17:27:03.892643 np0005474935.novalocal cinder-backup[104204]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=104204) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Oct 07 17:27:03.892914 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a', 0)] {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Oct 07 17:27:03.893581 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:03.907281 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.013s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:03.907505 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[e300881c-2efd-4aee-90e4-c6b88fd31fc9]: (4, ('38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a\n38.102.83.9:3260,4294967295 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:03.908246 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:03.926882 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:03.927025 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[50adc8a4-7443-473b-b45a-4618d3ef9272]: (4, ('tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash)\ntcp: [32] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a (non-flash)\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:03.927370 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('-m', 'session'): stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:27:03.927370 np0005474935.novalocal cinder-backup[104204]: tcp: [32] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a (non-flash) Oct 07 17:27:03.927370 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Oct 07 17:27:03.927575 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsi session list stdout=tcp: [29] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-325a0fe3-defe-4434-af3a-7f6788cdeea2 (non-flash) Oct 07 17:27:03.927575 np0005474935.novalocal cinder-backup[104204]: tcp: [32] 38.102.83.9:3260,1 iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a (non-flash) Oct 07 17:27:03.927575 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Oct 07 17:27:03.928277 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Resulting device map defaultdict(. at 0x7fe9d7f56d40>, {('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a'): ({'sdb'}, set())}) {{(pid=104204) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Oct 07 17:27:03.928525 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Removing single pathed devices sdb {{(pid=104204) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Oct 07 17:27:03.929207 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:03.938319 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[c127feb8-077e-4789-b6a0-2eb8fa1a3c0f]: [Errno 2] No such file or directory: 'multipathd' {{(pid=139586) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: Traceback (most recent call last): Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: ret = func(*f_args, **f_kwargs) Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: return func(*args, **kwargs) Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: return putils.execute(on_execute=on_execute, Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: obj = subprocess.Popen(cmd, Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: ^^^^^^^^^^^^^^^^^^^^^ Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: self._execute_child(args, executable, preexec_fn, close_fds, Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: raise child_exception_type(errno_num, err_msg, err_filename) Oct 07 17:27:03.941178 np0005474935.novalocal cinder-backup[104204]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Oct 07 17:27:03.942925 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[c127feb8-077e-4789-b6a0-2eb8fa1a3c0f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:03.943697 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Flushing IO for device /dev/sdb {{(pid=104204) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Oct 07 17:27:03.944857 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:03.956275 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.011s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:03.956755 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[46de268c-e1cc-4d3e-956d-4e38d93e4e99]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:03.956984 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=104204) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Oct 07 17:27:03.957711 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:04.021672 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.063s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:04.021997 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[77fa1209-e38f-4b5d-8a11-37efd06d7f6b]: (4, ('1', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:04.022323 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Oct 07 17:27:04.022526 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.linuxscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] SCSI volumes sdb have been removed. {{(pid=104204) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Oct 07 17:27:04.022993 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Disconnecting from: [('38.102.83.9:3260', 'iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a')] {{(pid=104204) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Oct 07 17:27:04.023885 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.startup -v manual {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:04.041874 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:04.042115 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[a253979a-219b-4b34-adb7-fc10a0e14aa2]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:04.042513 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:27:04.043352 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --logout {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:04.079473 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --logout" returned: 0 in 0.035s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:04.079778 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[223da771-494e-4618-a63c-8a736c371810]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a, portal: 38.102.83.9,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a, portal: 38.102.83.9,3260] successful.\n', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:04.080341 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a, portal: 38.102.83.9,3260] Oct 07 17:27:04.080341 np0005474935.novalocal cinder-backup[104204]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a, portal: 38.102.83.9,3260] successful. Oct 07 17:27:04.080341 np0005474935.novalocal cinder-backup[104204]: stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:27:04.081553 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op delete {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Oct 07 17:27:04.100698 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-127799f1-efe0-4649-baa6-c2d5e2634f2a -p 38.102.83.9:3260 --op delete" returned: 0 in 0.019s {{(pid=139586) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Oct 07 17:27:04.101050 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo.privsep.daemon [-] privsep: reply[330ae9ca-8f9e-4fb3-9cdb-f61db22fc601]: (4, ('', '')) {{(pid=139586) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Oct 07 17:27:04.101535 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=104204) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Oct 07 17:27:04.101955 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.base [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.223s {{(pid=104204) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Oct 07 17:27:04.102388 np0005474935.novalocal cinder-backup[104204]: DEBUG os_brick.initiator.connectors.iscsi [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] <== disconnect_volume: return (223ms) None {{(pid=104204) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Oct 07 17:27:07.671818 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-3ad297bf-8be8-4346-b285-9169774c4a73 tempest-VolumesBackupsTest-2000989888 None] Create backup finished. backup: 6f570b01-4615-4d3b-a9a4-3edc228cecc2. Oct 07 17:27:08.540863 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] Delete backup started, backup: 6f570b01-4615-4d3b-a9a4-3edc228cecc2. Oct 07 17:27:08.542992 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:27:08.543270 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:27:08.543612 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] delete started, backup: 6f570b01-4615-4d3b-a9a4-3edc228cecc2, container: tempest-volumesbackupstest-backup-container-1485026413, prefix: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:27:08.609788 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2_metadata', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:27:08.644135 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2_metadata in container: tempest-volumesbackupstest-backup-container-1485026413. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:08.670375 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212654/az_nova_backup_6f570b01-4615-4d3b-a9a4-3edc228cecc2_sha256file in container: tempest-volumesbackupstest-backup-container-1485026413. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:08.670606 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] delete 6f570b01-4615-4d3b-a9a4-3edc228cecc2 finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:27:08.673610 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:27:08.698694 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] Created reservations ['f100a7b4-b3e1-4cb6-9025-a212b23221e4', 'd2d8943d-b5c9-49bc-a643-b5d1696aa02f'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:27:08.737195 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-aadfa98f-50ec-4df2-b0a6-65ec8787346e tempest-VolumesBackupsTest-2000989888 None] Delete backup finished, backup 6f570b01-4615-4d3b-a9a4-3edc228cecc2 deleted. Oct 07 17:27:09.776787 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] Delete backup started, backup: eafcb882-ce54-4490-ba4a-8b026583da64. Oct 07 17:27:09.778117 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] Using swift URL https://38.102.83.9:8080/v1/AUTH_7124796c9a81406abe531b7e1755ce19 {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Oct 07 17:27:09.778403 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.drivers.swift [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] Connect to https://38.102.83.9:8080/v1/AUTH_ in "per_user" mode {{(pid=104204) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Oct 07 17:27:09.778703 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] delete started, backup: eafcb882-ce54-4490-ba4a-8b026583da64, container: tempest-volumesbackupstest-backup-container-1206883695, prefix: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Oct 07 17:27:09.829135 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:27:09.829322 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:27:09.855636 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] generated object list: ['volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00001', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00002', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00003', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00004', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00005', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00006', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00007', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00008', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00009', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00010', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00011', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00012', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00013', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00014', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00015', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00016', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00017', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00018', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00019', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00020', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00021', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_metadata', 'volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_sha256file']. {{(pid=104204) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Oct 07 17:27:09.883670 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00001 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:09.912261 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00002 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:09.940236 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00003 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:09.971598 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00004 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.001544 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00005 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.037304 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00006 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.073470 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00007 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.103245 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00008 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.136677 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00009 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.162881 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00010 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.189725 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00011 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.213885 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00012 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.236659 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00013 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.263438 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00014 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.295303 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00015 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.327667 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00016 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.359523 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00017 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.390907 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00018 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.422280 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00019 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.452360 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00020 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.481956 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64-00021 in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.511951 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_metadata in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.544408 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] deleted object: volume_325a0fe3-defe-4434-af3a-7f6788cdeea2/20251007212601/az_nova_backup_eafcb882-ce54-4490-ba4a-8b026583da64_sha256file in container: tempest-volumesbackupstest-backup-container-1206883695. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Oct 07 17:27:10.544784 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.backup.chunkeddriver [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] delete eafcb882-ce54-4490-ba4a-8b026583da64 finished. {{(pid=104204) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Oct 07 17:27:10.549961 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_db.sqlalchemy.engines [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=104204) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Oct 07 17:27:10.571696 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.quota [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] Created reservations ['ec0587ca-5e2e-49be-935d-ea12c388b0b4', '116ccea9-2f38-41a3-b834-c6fb415edc3f'] {{(pid=104204) reserve /opt/stack/cinder/cinder/quota.py:752}} Oct 07 17:27:10.592890 np0005474935.novalocal cinder-backup[104204]: INFO cinder.backup.manager [None req-1a6af6b4-42a8-4bd0-8b8f-5435db925668 tempest-VolumesBackupsTest-2000989888 None] Delete backup finished, backup eafcb882-ce54-4490-ba4a-8b026583da64 deleted. Oct 07 17:27:20.591137 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=46,cinder:UPDATE=19,cinder:INSERT=4 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:27:50.779469 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:28:09.828933 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:28:09.829582 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:29:09.831521 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:29:09.831521 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 07 17:29:50.783838 np0005474935.novalocal cinder-backup[104204]: DEBUG dbcounter [-] [104204] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=104204) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Oct 07 17:30:09.829436 np0005474935.novalocal cinder-backup[104204]: DEBUG oslo_service.periodic_task [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=104204) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Oct 07 17:30:09.829979 np0005474935.novalocal cinder-backup[104204]: DEBUG cinder.manager [None req-7140dd04-ceb7-4027-9a22-4c73469095e1 None None] Notifying Schedulers of capabilities ... {{(pid=104204) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}