Nov 21 15:30:19.444021 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:30:46.458522 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:30:46.458948 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:31:46.459239 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:31:46.459239 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:32:19.453801 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:32:46.461457 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:32:46.462106 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:33:46.460190 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:33:46.460431 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:34:19.449286 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:34:46.458694 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:34:46.458694 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:35:46.462046 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:35:46.462704 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:36:19.454659 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:36:46.469191 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:36:46.469572 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:37:46.469617 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:37:46.470195 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:38:19.464017 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:38:46.467789 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:38:46.467789 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:39:46.464443 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:39:46.464819 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:40:19.467379 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:40:46.468658 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:40:46.469098 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:41:46.473361 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:41:46.474759 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:42:19.476388 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:42:46.476415 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:42:46.476415 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:43:46.477704 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:43:46.477704 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:44:19.481848 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:44:46.479668 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:44:46.479953 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:44:48.364814 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Create backup started, backup: 9f8c3c75-3956-4a47-850b-927b0f04d9cd volume: 9c1a3edc-b2e4-4d0a-91b8-2e7927922a43 snapshot: 52aebd73-f9bf-4609-9b0c-ed19e9c204a3. Nov 21 15:44:48.369736 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumessnapshottestjson-backup-container-938716409',created_at=2025-11-21T20:44:48Z,data_timestamp=2025-11-21T20:44:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-2015029613',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=9f8c3c75-3956-4a47-850b-927b0f04d9cd,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='3b0b29d08f624beea07311e5d7808f9d',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id='52aebd73-f9bf-4609-9b0c-ed19e9c204a3',status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='da0e13bccf5b45dd9ed5da34c845ffea',volume_id=9c1a3edc-b2e4-4d0a-91b8-2e7927922a43) Nov 21 15:44:50.380253 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_3b0b29d08f624beea07311e5d7808f9d {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:44:50.380728 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:44:50.380728 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:44:50.382015 np0005531460.novalocal cinder-backup[107086]: INFO oslo.privsep.daemon [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp_cid6pme/privsep.sock'] Nov 21 15:44:50.399937 np0005531460.novalocal sudo[142843]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmp_cid6pme/privsep.sock Nov 21 15:44:51.178873 np0005531460.novalocal cinder-backup[107086]: INFO oslo.privsep.daemon [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Spawned new privsep daemon via rootwrap Nov 21 15:44:51.180297 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Accepted privsep connection to /tmp/tmp_cid6pme/privsep.sock {{(pid=107086) __init__ /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:377}} Nov 21 15:44:51.180297 np0005531460.novalocal cinder-backup[107086]: INFO oslo.privsep.daemon [-] privsep daemon starting Nov 21 15:44:51.180297 np0005531460.novalocal cinder-backup[107086]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Nov 21 15:44:51.180297 np0005531460.novalocal cinder-backup[107086]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_READ_SEARCH|CAP_SYS_ADMIN/CAP_DAC_READ_SEARCH|CAP_SYS_ADMIN/none Nov 21 15:44:51.180297 np0005531460.novalocal cinder-backup[107086]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 142847 Nov 21 15:44:51.181385 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[49634e05-6593-4fce-bba5-a9d772539601]: (2,) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:51.285259 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:51.291058 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.006s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:51.291282 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[912d5d54-18c2-483b-8d60-43f8350ad467]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[bfc4c371-bb80-49d3-9ef4-1316b91fd4da]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:51.293866 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:44:51.294178 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfc4c371-bb80-49d3-9ef4-1316b91fd4da]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:51.294972 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:44:51.295394 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:44:51.295828 np0005531460.novalocal cinder-backup[107086]: WARNING os_brick.initiator.connectors.nvmeof [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath' Nov 21 15:44:51.296561 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:51.304434 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.007s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:51.304434 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d697798d-4caf-44e9-81ee-6b44de67c9c4]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:51.304973 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:51.311540 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.006s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:51.311938 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[22714b45-9deb-493c-b78b-27543bd3ec68]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:51.313194 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3193b088-7158-4dd2-a68b-6603252b0bb9]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:51.313906 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:51.325792 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:44:51.325792 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:44:51.326302 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:44:51.327638 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194'] Nov 21 15:44:51.328027 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:44:51.328365 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:44:51.328993 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] <== get_connector_properties: return (947ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:44:54.851886 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:44:54.853509 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6', 'target_lun': 0, 'volume_id': '10a25cff-f1f3-44d5-aded-232f3c3034f6', 'auth_method': 'CHAP', 'auth_username': 'KvSuEMb4THo6PNCwZwQF', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:44:54.853947 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:44:54.854669 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:44:54.854977 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6', 'target_lun': 0, 'volume_id': '10a25cff-f1f3-44d5-aded-232f3c3034f6', 'auth_method': 'CHAP', 'auth_username': 'KvSuEMb4THo6PNCwZwQF', 'auth_password': '2iPSf6JBdDeftBJV', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:44:54.856626 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:54.865207 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d34a42fe-70e0-4348-ad45-d93a5c4f2c0a]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:44:54.869933 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:44:54.871678 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d34a42fe-70e0-4348-ad45-d93a5c4f2c0a]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:54.872577 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:44:54.873191 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:44:54.873406 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:44:54.874329 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:54.885157 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:54.885371 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9117b2b-c87b-4577-9539-9ad2476e3479]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:54.885755 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:44:54.885755 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:54.886821 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:54.900111 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:54.900338 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a688af9-0135-4fa8-b67a-b3f0775c4c0c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:54.900790 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6] added Nov 21 15:44:54.900790 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:44:54.900790 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:44:54.900790 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:44:54.900790 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:54.901047 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:44:54.901666 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.028(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:44:54.902034 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:44:55.903146 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:44:55.904335 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.915375 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.915708 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[806e89ba-97a5-41f2-be32-5464ed934e7f]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:44:55.917364 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:55.920068 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.927727 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.928045 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c12a936c-2dca-422f-b0d0-a185024f3e69]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.928432 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:55.929227 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d63c12e-72b1-4272-a53c-8323809497d5]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v KvSuEMb4THo6PNCwZwQF {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v KvSuEMb4THo6PNCwZwQF" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9b312a3-4816-4a30-be17-04e2b5639591]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'KvSuEMb4THo6PNCwZwQF'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b09f8bb6-37e3-4f29-abd0-01e400dd0cb3]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:55.968866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.970641 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[271c0896-f072-462c-8446-a8681a20c2a9]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: WARNING os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 21 15:44:55.972679 np0005531460.novalocal cinder-backup[107086]: Nov 21 15:44:55.972938 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:55.991448 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --login" returned: 0 in 0.017s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:55.991626 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bca217d-e3fc-49a0-80e2-f9ec137e331b]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:55.992109 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6, portal: 38.102.83.94,3260] successful. Nov 21 15:44:55.992109 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:55.993065 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:56.005111 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:56.005278 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe68428b-6007-4eb3-8acf-1f79a96c178e]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:56.005651 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:44:56.006418 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:56.016760 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:56.017124 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[83c470b4-8f15-4ec3-808e-79b2924eef78]: (4, ('tcp: [18] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:56.017504 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('-m', 'session'): stdout=tcp: [18] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 (non-flash) Nov 21 15:44:56.017504 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:44:56.017734 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsi session list stdout=tcp: [18] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 (non-flash) Nov 21 15:44:56.017734 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:44:56.018420 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.145s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:44:56.018677 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:44:56.020258 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] HCTL ('2', '-', '-', 0) found on session 18 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:44:56.020553 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:44:56.021332 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:56.047760 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.024s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:56.047760 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[65c0ef2c-2c57-41dd-aa77-4814209402c7]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:44:56.047760 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Searching for a device in session 18 and hctl ['2', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:44:57.049000 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Searching for a device in session 18 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:44:57.049816 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6', 'target_lun': 0, 'volume_id': '10a25cff-f1f3-44d5-aded-232f3c3034f6', 'auth_method': 'CHAP', 'auth_username': 'KvSuEMb4THo6PNCwZwQF', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:44:57.050887 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.196s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:44:57.051231 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] <== connect_volume: return (2197ms) {'type': 'block', 'scsi_wwn': '360014055c3275e1745646dc83111c2cb', 'path': '/dev/sda'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:44:57.051915 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:44:57.083330 np0005531460.novalocal sudo[142973]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 21 15:44:57.439209 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.386s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:44:57.450385 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:44:57.462601 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] _create_container started, container: tempest-volumessnapshottestjson-backup-container-938716409,backup: 9f8c3c75-3956-4a47-850b-927b0f04d9cd. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:44:57.704802 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] generate_object_name_prefix: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:44:57.711313 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] starting backup of volume: 9c1a3edc-b2e4-4d0a-91b8-2e7927922a43, volume size: 1073741824, object names prefix volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:44:57.885173 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:44:58.030102 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:44:58.030509 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:44:58.221615 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:44:58.222100 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:44:58.413539 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:44:58.548123 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:44:58.548400 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:44:58.708320 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:44:58.708320 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:44:58.878144 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:44:59.008874 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:44:59.009055 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:44:59.154199 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:44:59.154462 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:44:59.333438 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:44:59.472698 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:44:59.472905 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:44:59.620383 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:44:59.620697 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:44:59.807518 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:44:59.946961 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:44:59.946961 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:00.087545 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:00.087836 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:00.255797 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:00.377197 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:00.377732 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:00.572590 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:00.572758 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:00.996409 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:01.221500 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:01.221749 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:01.365925 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:01.366195 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:01.789250 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:01.938272 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:01.938534 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:02.049623 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:02.049745 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:02.211927 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:02.365935 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:02.366280 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:02.519668 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:02.519830 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:02.691600 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:02.823499 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:02.823972 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:02.981083 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:02.981545 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:03.144708 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:03.249029 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:03.250858 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:03.411624 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:03.412000 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:03.593471 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:03.688650 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:03.688958 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:03.848527 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:03.848722 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:04.230003 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:04.349311 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:04.349561 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:04.466419 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:04.466593 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:04.838277 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:04.992214 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:04.992482 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:05.101130 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:05.101388 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:05.671794 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:05.892651 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:05.892997 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:06.073799 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:06.074132 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:06.216741 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:06.367322 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:06.367449 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:06.517558 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:06.517682 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:06.699847 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:06.797247 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:06.797247 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:06.913033 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:06.913203 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:07.098671 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:07.249249 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:07.249376 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:07.400138 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:07.400412 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:07.855821 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:07.982408 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:07.982807 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:08.097797 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:08.099148 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:08.458206 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:08.564380 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:08.564643 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:08.704962 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:08.705084 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:08.768591 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:45:08.828195 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:45:08.828351 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:45:08.909195 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup MD5 for volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:45:08.909787 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:45:08.915270 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:45:08.925418 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:45:08.925558 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:45:08.929845 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:45:08.929974 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:45:08.935293 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:45:08.935540 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] _write_sha256file started, container name: tempest-volumessnapshottestjson-backup-container-938716409, sha256file filename: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:45:09.009998 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:45:09.011046 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] _write_metadata started, container name: tempest-volumessnapshottestjson-backup-container-938716409, metadata filename: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:45:09.045497 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "9f8c3c75-3956-4a47-850b-927b0f04d9cd",\n "backup_name": "tempest-VolumesSnapshotTestJSON-Backup-2015029613",\n "created_at": "2025-11-21 20:44:48+00:00",\n "objects": [\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "9c1a3edc-b2e4-4d0a-91b8-2e7927922a43",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"9c1a3edc-b2e4-4d0a-91b8-2e7927922a43\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"da0e13bccf5b45dd9ed5da34c845ffea\\", \\"project_id\\": \\"3b0b29d08f624beea07311e5d7808f9d\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"available\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:44:45.000000\\", \\"launched_at\\": \\"2025-11-21T20:44:46.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesSnapshotTestJSON-Volume-1948047061\\", \\"display_description\\": null, \\"provider_location\\": null, \\"provider_auth\\": null, \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": null, \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:44:45.000000\\", \\"updated_at\\": \\"2025-11-21T20:44:46.000000\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:45:09.055792 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] backup 9f8c3c75-3956-4a47-850b-927b0f04d9cd finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:45:09.133321 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.161328 np0005531460.novalocal sudo[143237]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 21 15:45:09.456563 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.323s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.457429 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6', 'target_lun': 0, 'volume_id': '10a25cff-f1f3-44d5-aded-232f3c3034f6', 'auth_method': 'CHAP', 'auth_username': 'KvSuEMb4THo6PNCwZwQF', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014055c3275e1745646dc83111c2cb', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:45:09.457719 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:45:09.458254 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:45:09.459380 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.467621 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.467792 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[050f1398-2cd0-4c70-b684-0cd0306752be]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:45:09.468080 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:45:09.468367 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:45:09.468367 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:45:09.468367 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:45:09.468709 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:45:09.469385 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.477313 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.477593 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[543ccc05-d584-4258-86a3-c4cd49823553]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.478238 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d262b842-8b64-498a-bff9-c77e9bddbf00]: (4, ('tcp: [18] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('-m', 'session'): stdout=tcp: [18] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 (non-flash) Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsi session list stdout=tcp: [18] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 (non-flash) Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:45:09.490815 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Resulting device map defaultdict(. at 0x7fddd67b7ec0>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6'): ({'sda'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:45:09.491139 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Removing single pathed devices sda {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:45:09.491560 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.498902 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a3f8f252-509b-44f8-8aee-b6a71d055acc]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:45:09.499699 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:45:09.501469 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3f8f252-509b-44f8-8aee-b6a71d055acc]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.501879 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Flushing IO for device /dev/sda {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:45:09.502438 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.508898 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.006s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.509232 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ec53703-fe1b-474b-aab9-15786a57338b]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.509418 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:45:09.510006 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.551107 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.040s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.551478 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[96bbc331-69c6-4877-8c57-b5cfc26c8b21]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.552132 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:45:09.552432 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:45:09.552880 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:45:09.554150 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.563027 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.563286 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[230dcbac-89c0-414a-9b48-36b072eae983]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.563672 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:45:09.565183 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.589247 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --logout" returned: 0 in 0.024s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.590015 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa21421f-51cc-429c-8994-238608ec11c7]: (4, ('Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6, portal: 38.102.83.94,3260]\nLogout of [sid: 18, target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.590487 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6, portal: 38.102.83.94,3260] Nov 21 15:45:09.590487 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 18, target: iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6, portal: 38.102.83.94,3260] successful. Nov 21 15:45:09.590487 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:45:09.592289 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:45:09.600641 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-10a25cff-f1f3-44d5-aded-232f3c3034f6 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:45:09.600930 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2202eaa4-7edc-4a81-9a34-7959d6989ff3]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:45:09.601339 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:45:09.601743 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.143s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:45:09.602491 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] <== disconnect_volume: return (144ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:45:13.171473 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1cce3c78-cc79-4aa5-b867-55126a795113 tempest-VolumesSnapshotTestJSON-591229851 None] Create backup finished. backup: 9f8c3c75-3956-4a47-850b-927b0f04d9cd. Nov 21 15:45:14.158586 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] Delete backup started, backup: 9f8c3c75-3956-4a47-850b-927b0f04d9cd. Nov 21 15:45:14.160479 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_3b0b29d08f624beea07311e5d7808f9d {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:45:14.160756 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:45:14.161235 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] delete started, backup: 9f8c3c75-3956-4a47-850b-927b0f04d9cd, container: tempest-volumessnapshottestjson-backup-container-938716409, prefix: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:45:14.226058 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] generated object list: ['volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00001', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00002', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00003', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00004', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00005', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00006', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00007', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00008', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00009', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00010', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00011', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00012', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00013', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00014', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00015', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00016', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00017', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00018', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00019', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00020', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00021', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd_metadata', 'volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:45:14.254860 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00001 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.290561 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00002 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.321554 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00003 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.349850 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00004 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.377943 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00005 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.402912 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00006 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.427255 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00007 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.453617 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00008 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.484032 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00009 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.520486 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00010 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.552043 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00011 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.580850 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00012 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.612739 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00013 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.641322 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00014 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.670288 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00015 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.699608 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00016 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.727466 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00017 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.757412 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00018 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.806699 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00019 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.845193 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00020 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.875437 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd-00021 in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.899803 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd_metadata in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.923491 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] deleted object: volume_9c1a3edc-b2e4-4d0a-91b8-2e7927922a43/20251121204457/az_nova_backup_9f8c3c75-3956-4a47-850b-927b0f04d9cd_sha256file in container: tempest-volumessnapshottestjson-backup-container-938716409. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:45:14.923662 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] delete 9f8c3c75-3956-4a47-850b-927b0f04d9cd finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:45:14.926994 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:45:14.957026 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] Created reservations ['49af61f0-ae23-44f0-9345-a7805f5e1289', '70fb32cd-3a57-48dd-8838-f6dc47aca275'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:45:14.987423 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-ce2e59fb-f6bf-48cb-b8a4-7ee5370bd79f tempest-VolumesSnapshotTestJSON-591229851 None] Delete backup finished, backup 9f8c3c75-3956-4a47-850b-927b0f04d9cd deleted. Nov 21 15:45:24.986939 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=41,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:45:46.486092 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:45:46.486781 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:46:19.481072 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:46:46.494278 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:46:46.494700 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:47:00.775639 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Create backup started, backup: ef556dbc-2857-4a84-aa19-e41f0f153047 volume: bdef1043-6af8-4f7e-a163-b9104519911a. Nov 21 15:47:00.779720 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1629482955',created_at=2025-11-21T20:47:00Z,data_timestamp=2025-11-21T20:47:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1979407560',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=ef556dbc-2857-4a84-aa19-e41f0f153047,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='99dea2ee54874291a7a3284a115f81e4',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='78aab0b8a676406dab9c030d6a4b429f',volume_id=bdef1043-6af8-4f7e-a163-b9104519911a) Nov 21 15:47:02.308517 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:47:02.309119 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:47:02.309423 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:47:02.310909 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:02.320183 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:02.320530 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[659efe11-0ae7-41f9-aa7e-316ddf9c0412]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f0eaf396-526c-44ba-a407-e8f43cdd50ed]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:02.322835 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:47:02.323491 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0eaf396-526c-44ba-a407-e8f43cdd50ed]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:02.324143 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:47:02.324588 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:47:02.325432 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:02.335858 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:02.336088 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc76173c-2ef6-4dde-93bd-ab7e2e74e6d2]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:02.337256 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:02.348197 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:02.351468 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb1f3d57-d7e4-4101-89a1-2ee32f01a2c0]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:02.352512 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[eefb993f-f12b-4c8d-93df-e19a683ba790]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:02.353034 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:02.370809 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:47:02.371235 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:47:02.373545 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:47:02.375088 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194', 'fe80::fc16:3eff:febe:2819', 'fe80::fc16:3eff:fec9:cff8', 'fe80::fc16:3eff:feae:154a', 'fe80::fc16:3eff:fe73:c471', 'fe80::1472:7bff:fe85:77b', 'fe80::fc16:3eff:fec4:36a5', 'fe80::7c9a:f1ff:fe24:dea3'] Nov 21 15:47:02.375396 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:47:02.375651 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:47:02.376316 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (66ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194', 'fe80::fc16:3eff:febe:2819', 'fe80::fc16:3eff:fec9:cff8', 'fe80::fc16:3eff:feae:154a', 'fe80::fc16:3eff:fe73:c471', 'fe80::1472:7bff:fe85:77b', 'fe80::fc16:3eff:fec4:36a5', 'fe80::7c9a:f1ff:fe24:dea3']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:47:05.888679 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:47:05.891819 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4', 'target_lun': 0, 'volume_id': '25c0d3bc-23ea-4195-85db-6d6b326990b4', 'auth_method': 'CHAP', 'auth_username': 'YdoLJPSg8zViNiJbGY2r', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:47:05.891819 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:47:05.892620 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:47:05.893371 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4', 'target_lun': 0, 'volume_id': '25c0d3bc-23ea-4195-85db-6d6b326990b4', 'auth_method': 'CHAP', 'auth_username': 'YdoLJPSg8zViNiJbGY2r', 'auth_password': '3CWAYwXAgcf6Wmqd', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:47:05.895545 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:05.904832 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[3c11fe1e-ff82-46e6-acdd-71037ae63542]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:47:05.907816 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:47:05.911630 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c11fe1e-ff82-46e6-acdd-71037ae63542]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:05.914084 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:47:05.914719 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:47:05.919156 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:47:05.919800 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:05.931894 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:05.931894 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6917e81e-4e1c-4082-b4cd-43a12088ee52]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:05.931894 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:47:05.931894 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:05.932522 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[24cadc14-0a68-43b9-a92f-91a789084fe1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4] added Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.033(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:47:05.950170 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:47:06.951649 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:47:06.952924 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:06.969235 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260" returned: 0 in 0.016s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:06.969660 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6549abaa-db75-426f-a284-fe9b6fad5124]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:47:06.972249 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:06.975740 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:06.987620 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:06.987620 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d7c5b4d-f538-43b6-b211-a197758714af]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:06.987620 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:06.990393 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.001965 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.002369 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c33c52ea-71b5-4905-84a0-e888fe29f323]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.002977 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:07.003977 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v YdoLJPSg8zViNiJbGY2r {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.015606 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v YdoLJPSg8zViNiJbGY2r" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.016926 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c18019d-e914-478b-8191-95830a7bb8ab]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.016926 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YdoLJPSg8zViNiJbGY2r'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:07.017669 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.029847 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.029847 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e647fad-9707-49a6-8457-cbb2f85a84c0]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.029847 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:07.030299 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.065474 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.066001 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cbc0a24-6f30-433c-a597-9409ea1b135e]: (4, ('tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash)\ntcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.066713 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash) Nov 21 15:47:07.066713 np0005531460.novalocal cinder-backup[107086]: tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:07.066713 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:47:07.067471 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash) Nov 21 15:47:07.067471 np0005531460.novalocal cinder-backup[107086]: tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:07.067471 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:47:07.068687 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.092920 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --login" returned: 0 in 0.022s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.092920 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5031fc03-6f35-4f96-9f84-5b1a2788db87]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.092920 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4, portal: 38.102.83.94,3260] successful. Nov 21 15:47:07.092920 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:07.093543 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.106031 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.106311 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fd9d1d2-de52-488b-aa6d-4f45b294147b]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.106596 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:07.107367 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.121421 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.121831 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[afbc46c7-daf6-4956-87d6-542e5dc93fea]: (4, ('tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash)\ntcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash)\ntcp: [22] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.122078 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash) Nov 21 15:47:07.122078 np0005531460.novalocal cinder-backup[107086]: tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:07.122078 np0005531460.novalocal cinder-backup[107086]: tcp: [22] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 (non-flash) Nov 21 15:47:07.122078 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:47:07.122227 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash) Nov 21 15:47:07.122227 np0005531460.novalocal cinder-backup[107086]: tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:07.122227 np0005531460.novalocal cinder-backup[107086]: tcp: [22] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 (non-flash) Nov 21 15:47:07.122227 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:47:07.122594 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.208s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:47:07.122801 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:47:07.123406 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] HCTL ('4', '-', '-', 0) found on session 22 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:47:07.123564 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Scanning host 4 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:47:07.124301 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host4/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:07.151734 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host4/scan" returned: 0 in 0.026s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:07.152107 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2820bd2-4204-4d2a-9765-a5b3f1ede3e5]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:07.153907 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 22 and hctl ['4', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:47:08.155815 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 22 and hctl ['4', '*', '*', 0] yield: sdc {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:47:08.156665 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Connected to sdc using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4', 'target_lun': 0, 'volume_id': '25c0d3bc-23ea-4195-85db-6d6b326990b4', 'auth_method': 'CHAP', 'auth_username': 'YdoLJPSg8zViNiJbGY2r', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:47:08.157527 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.265s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:47:08.158053 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2267ms) {'type': 'block', 'scsi_wwn': '36001405c2207994539f4710b5fecec73', 'path': '/dev/sdc'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:47:08.159000 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdc {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:08.196949 np0005531460.novalocal sudo[145500]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdc Nov 21 15:47:08.518178 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdc" returned: 0 in 0.359s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:08.523013 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:47:08.532386 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1629482955,backup: ef556dbc-2857-4a84-aa19-e41f0f153047. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:47:08.627636 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] generate_object_name_prefix: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047 {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:47:08.631692 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] starting backup of volume: bdef1043-6af8-4f7e-a163-b9104519911a, volume size: 1073741824, object names prefix volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:47:08.812978 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:08.949515 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:08.949818 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:09.115825 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:09.115825 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:09.386094 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:09.549970 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:09.550370 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:09.680472 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:09.680730 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:09.919699 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:10.052257 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:10.052376 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:10.173783 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:10.173929 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:10.314571 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:10.487145 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:10.487561 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:10.612449 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:10.612662 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:10.961704 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:11.094364 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:11.094422 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:11.249013 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:11.249285 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:11.476196 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:11.624790 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:11.625645 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:11.783076 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:11.783198 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:12.132569 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:12.289044 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:12.289352 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:12.426994 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:12.427093 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:12.588949 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:12.721524 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:12.722199 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:12.873533 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:12.873665 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:13.231439 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:13.364437 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:13.364698 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:13.543147 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:13.543318 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:13.933268 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:14.068695 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:14.068934 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:14.260371 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:14.260594 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:14.497000 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:14.632649 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:14.632951 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:14.786572 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:14.787471 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:14.922294 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:15.019038 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:15.019571 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:15.177508 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:15.177823 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:15.385292 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:15.509388 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:15.509566 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:15.634941 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:15.635112 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:16.026582 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:16.160541 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:16.160707 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:16.310753 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:16.310753 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:16.504008 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:16.666714 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:16.666788 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:16.852267 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:16.853594 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:16.979592 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:17.107219 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:17.107819 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:17.221684 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:17.221944 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:17.617062 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:17.842184 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:17.842451 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:17.982687 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:17.982984 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:18.204945 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:18.354854 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:18.354854 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:18.499274 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:18.499546 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:18.674906 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:18.877248 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:18.877248 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:19.026829 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:19.026829 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:19.163792 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:19.344639 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:19.345330 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:19.481600 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:19.481895 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:19.569449 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:19.641020 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:19.641020 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:19.742370 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:19.742494 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:19.747909 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:47:19.756152 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:47:19.756293 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:47:19.759046 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:47:19.759176 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:47:19.761421 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:47:19.761612 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1629482955, sha256file filename: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:47:19.841925 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:47:19.842970 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1629482955, metadata filename: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:47:19.881995 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "ef556dbc-2857-4a84-aa19-e41f0f153047",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1979407560",\n "created_at": "2025-11-21 20:47:00+00:00",\n "objects": [\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "bdef1043-6af8-4f7e-a163-b9104519911a",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"bdef1043-6af8-4f7e-a163-b9104519911a\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"78aab0b8a676406dab9c030d6a4b429f\\", \\"project_id\\": \\"99dea2ee54874291a7a3284a115f81e4\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:46:11.000000\\", \\"launched_at\\": \\"2025-11-21T20:46:11.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1911093223\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a 0\\", \\"provider_auth\\": \\"CHAP RvK2MYHTRyJNBDqcD5AE dNrz5C29RTpx7uc8\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:46:10.000000\\", \\"updated_at\\": \\"2025-11-21T20:47:00.000000\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:47:19.888848 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] backup ef556dbc-2857-4a84-aa19-e41f0f153047 finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:47:19.983397 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdc {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.020321 np0005531460.novalocal sudo[145653]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdc Nov 21 15:47:20.451568 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdc" returned: 0 in 0.468s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.453343 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4', 'target_lun': 0, 'volume_id': '25c0d3bc-23ea-4195-85db-6d6b326990b4', 'auth_method': 'CHAP', 'auth_username': 'YdoLJPSg8zViNiJbGY2r', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405c2207994539f4710b5fecec73', 'path': '/dev/sdc'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:47:20.453658 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:47:20.456205 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:47:20.457363 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.471674 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.472092 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d363d24-85c4-4c98-a704-2c01cfc870a9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:47:20.472455 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:47:20.472961 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:47:20.472961 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:47:20.472961 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:47:20.473055 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:47:20.473619 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.487330 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.487330 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c6873cf-0629-4b88-a9ac-b1b363e00e83]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12\n38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4\n38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.487486 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.500982 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.501220 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8957e31-6161-4af6-b904-b953de508df3]: (4, ('tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash)\ntcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash)\ntcp: [22] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.501441 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash) Nov 21 15:47:20.501441 np0005531460.novalocal cinder-backup[107086]: tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:20.501441 np0005531460.novalocal cinder-backup[107086]: tcp: [22] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 (non-flash) Nov 21 15:47:20.501441 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:47:20.501599 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [20] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-bdef1043-6af8-4f7e-a163-b9104519911a (non-flash) Nov 21 15:47:20.501599 np0005531460.novalocal cinder-backup[107086]: tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:20.501599 np0005531460.novalocal cinder-backup[107086]: tcp: [22] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 (non-flash) Nov 21 15:47:20.501599 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:47:20.502349 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb83cf100>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4'): ({'sdc'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:47:20.502543 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sdc {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:47:20.504567 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.513414 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d76d2357-5a74-4e0d-bc99-95211e78cc1e]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:47:20.513651 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:47:20.514283 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d76d2357-5a74-4e0d-bc99-95211e78cc1e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.514283 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sdc {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:47:20.516122 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.524386 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.524859 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[af56962d-cb15-437a-938b-12d20e5280c9]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.525059 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:47:20.526492 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.573883 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.048s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.573974 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8af8ece2-78b6-472c-af34-d1150dd3165e]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.574232 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sdc have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:47:20.574420 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] SCSI volumes sdc have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:47:20.577328 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:47:20.578208 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.591362 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.591491 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3f321a0-aa80-4c2c-b397-cedb5f7e713a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.592035 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:20.592761 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.632518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --logout" returned: 0 in 0.039s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.632518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[28209044-d952-4e89-96f6-30fc8c225696]: (4, ('Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4, portal: 38.102.83.94,3260]\nLogout of [sid: 22, target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.633800 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4, portal: 38.102.83.94,3260] Nov 21 15:47:20.633800 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 22, target: iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4, portal: 38.102.83.94,3260] successful. Nov 21 15:47:20.633800 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:20.634729 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:20.649456 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-25c0d3bc-23ea-4195-85db-6d6b326990b4 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:20.651067 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7061ac4a-7cd3-490e-a6ef-b3433dad9bd6]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:20.651067 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:20.651067 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.196s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:47:20.651244 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (197ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:47:25.692416 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-72ed889e-d455-43c3-bd79-8beac27f5757 tempest-VolumesBackupsTest-633227990 None] Create backup finished. backup: ef556dbc-2857-4a84-aa19-e41f0f153047. Nov 21 15:47:26.747350 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] Delete backup started, backup: ef556dbc-2857-4a84-aa19-e41f0f153047. Nov 21 15:47:26.748925 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:47:26.749219 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:47:26.749583 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] delete started, backup: ef556dbc-2857-4a84-aa19-e41f0f153047, container: tempest-volumesbackupstest-backup-container-1629482955, prefix: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:47:26.841985 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00001', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00002', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00003', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00004', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00005', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00006', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00007', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00008', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00009', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00010', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00011', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00012', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00013', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00014', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00015', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00016', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00017', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00018', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00019', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00020', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00021', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047_metadata', 'volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:47:26.878293 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00001 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:26.910020 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00002 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:26.944605 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00003 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:26.971872 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00004 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:26.997643 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00005 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.021739 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00006 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.048811 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00007 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.079916 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00008 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.109893 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00009 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.143951 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00010 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.176001 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00011 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.204989 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00012 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.232637 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00013 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.260160 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00014 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.297494 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00015 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.334168 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00016 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.362552 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00017 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.397895 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00018 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.428559 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00019 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.459546 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00020 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.493009 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047-00021 in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.526624 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047_metadata in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.558680 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] deleted object: volume_bdef1043-6af8-4f7e-a163-b9104519911a/20251121204708/az_nova_backup_ef556dbc-2857-4a84-aa19-e41f0f153047_sha256file in container: tempest-volumesbackupstest-backup-container-1629482955. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:47:27.559074 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] delete ef556dbc-2857-4a84-aa19-e41f0f153047 finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:47:27.564799 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:47:27.601991 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] Created reservations ['c0195c06-83c9-4cc0-82f0-91e924792746', '12ececfe-33af-491c-9d2f-84a4e0997f4d'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:47:27.625117 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-40aae721-fdc1-46fe-927c-e12fb2330f5b tempest-VolumesBackupsTest-633227990 None] Delete backup finished, backup ef556dbc-2857-4a84-aa19-e41f0f153047 deleted. Nov 21 15:47:37.623861 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=41,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:47:46.357258 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Create backup started, backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a volume: 3750d5f1-2033-482d-a330-fc1c3f08b018. Nov 21 15:47:46.360622 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1461043610',created_at=2025-11-21T20:47:46Z,data_timestamp=2025-11-21T20:47:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-699357810',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=cefd6587-d307-41e5-9c94-3ccd62e93a0a,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='99dea2ee54874291a7a3284a115f81e4',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='78aab0b8a676406dab9c030d6a4b429f',volume_id=3750d5f1-2033-482d-a330-fc1c3f08b018) Nov 21 15:47:46.389941 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:47:46.390045 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:47:46.390361 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:47:46.392274 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:46.399450 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:46.400157 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f3cfe49-da5e-422e-918b-3e522f204823]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d054b359-b27d-49ad-94ef-034733d9e561]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:46.401365 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:47:46.401922 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d054b359-b27d-49ad-94ef-034733d9e561]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:46.402193 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:47:46.402788 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:47:46.403619 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:46.412997 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:46.413211 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd3be6ca-de37-4415-a11d-e48b5b80c3c9]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:46.414153 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:46.424097 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:46.424312 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebb5a87f-9b6d-4ec5-8ef6-4b69d1bdf13a]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:46.425068 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e958f29-b552-40b5-b916-82bc7f6b5261]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:46.425522 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:46.440878 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:47:46.442236 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:47:46.443407 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:47:46.444674 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194', 'fe80::fc16:3eff:febe:2819', 'fe80::fc16:3eff:fec9:cff8', 'fe80::fc16:3eff:fec4:36a5', 'fe80::7c9a:f1ff:fe24:dea3', 'fe80::fc16:3eff:feae:154a'] Nov 21 15:47:46.445014 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:47:46.445214 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:47:46.445729 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (54ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194', 'fe80::fc16:3eff:febe:2819', 'fe80::fc16:3eff:fec9:cff8', 'fe80::fc16:3eff:fec4:36a5', 'fe80::7c9a:f1ff:fe24:dea3', 'fe80::fc16:3eff:feae:154a']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:47:46.491535 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:47:46.491717 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:47:49.956919 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:47:49.957806 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018', 'target_lun': 0, 'volume_id': '3750d5f1-2033-482d-a330-fc1c3f08b018', 'auth_method': 'CHAP', 'auth_username': 'NAKNU4xYFLjMjZbV6BVE', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:47:49.958134 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:47:49.958846 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:47:49.959079 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018', 'target_lun': 0, 'volume_id': '3750d5f1-2033-482d-a330-fc1c3f08b018', 'auth_method': 'CHAP', 'auth_username': 'NAKNU4xYFLjMjZbV6BVE', 'auth_password': 'h4uoJMg2QvUjE8aK', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:47:49.960413 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:49.969614 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[68e236c1-7970-4cd1-ae26-f2554185501b]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:47:49.970347 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:47:49.972101 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[68e236c1-7970-4cd1-ae26-f2554185501b]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:49.973408 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:47:49.974376 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:47:49.974802 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:47:49.976206 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:49.986711 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260" returned: 21 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:49.987028 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[34b7f3e1-145a-483c-9fea-4aafca11aba3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:49.987488 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:47:49.987488 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:49.988582 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:49.999923 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:50.000250 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[64e8a13e-12ee-466b-be83-d3924302d5a5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:50.000744 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018] added Nov 21 15:47:50.000744 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:47:50.000744 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:47:50.000744 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:47:50.000744 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:50.001078 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:47:50.001927 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.027(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:47:50.002223 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:47:51.003742 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:47:51.005452 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.016710 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.017156 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c65ee6f-84c7-4c1a-b191-af976f71eb26]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:47:51.018898 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.021673 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.031176 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.031433 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec5447cc-a4c3-461c-8680-91846b9015c5]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.031879 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.032848 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.048249 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.048472 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[38857787-ec63-4ac7-a05d-6587a0a0a0c6]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.048886 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.049948 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v NAKNU4xYFLjMjZbV6BVE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.062991 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v NAKNU4xYFLjMjZbV6BVE" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.063261 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[66a524df-d886-49c2-b5c5-18f28783ee3d]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.063651 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NAKNU4xYFLjMjZbV6BVE'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.064816 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.076078 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.076335 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[429dd3da-424a-47e2-a381-cf7a8f0b7ae5]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.076858 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.077785 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.089627 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.089915 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[56f3c82f-077c-4ce8-82bc-de74b7d3f51f]: (4, ('tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.090303 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:51.090303 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:47:51.090588 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:51.090588 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:47:51.091546 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.113576 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --login" returned: 0 in 0.021s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.113811 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd14f263-4606-4b79-aadd-a68153e644dd]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.115620 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018, portal: 38.102.83.94,3260] successful. Nov 21 15:47:51.115620 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.118173 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.129959 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.130361 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3eb7f4fa-d153-4564-99dd-7338a6b4760c]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.131146 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:47:51.132356 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.145281 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.145641 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8a143ac-49e3-47c2-b9b2-94df6d8a1194]: (4, ('tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash)\ntcp: [24] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: tcp: [24] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 (non-flash) Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [21] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1f31334b-da3d-4de9-bb6e-48cc28c10b12 (non-flash) Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: tcp: [24] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 (non-flash) Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:47:51.147497 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.173s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:47:51.147842 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:47:51.148555 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] HCTL ('2', '-', '-', 0) found on session 24 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:47:51.148887 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:47:51.149874 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:51.177213 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.026s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:51.177694 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b6f7863-fbdd-43ae-a7dc-0ba6e540c8f8]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:47:51.178837 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 24 and hctl ['2', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:47:52.180832 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 24 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:47:52.181665 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018', 'target_lun': 0, 'volume_id': '3750d5f1-2033-482d-a330-fc1c3f08b018', 'auth_method': 'CHAP', 'auth_username': 'NAKNU4xYFLjMjZbV6BVE', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:47:52.182495 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.224s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:47:52.182853 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2224ms) {'type': 'block', 'scsi_wwn': '36001405e114ca4236d04fafb184eaef5', 'path': '/dev/sda'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:47:52.183478 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:47:52.210399 np0005531460.novalocal sudo[146462]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 21 15:47:52.537092 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.353s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:47:52.544555 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:47:52.555622 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1461043610,backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:47:52.631908 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] generate_object_name_prefix: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:47:52.637880 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] starting backup of volume: 3750d5f1-2033-482d-a330-fc1c3f08b018, volume size: 1073741824, object names prefix volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:47:52.868928 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:53.388095 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 19857195 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:53.388641 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:53.726925 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001: 82571f7121cb9475563edde615e4e4df {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:53.727193 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:53.945384 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:54.051052 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:54.051052 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:54.203915 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:54.203915 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:54.400227 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:54.538102 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 1362431 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:54.538384 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:54.661655 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003: a35891500449df17e893f84bd88472fb {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:54.661986 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:54.851668 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:54.986537 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:54.986746 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:55.134886 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:55.135071 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:55.313342 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:55.432622 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:55.432993 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:55.585346 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:55.585661 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:55.759102 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:55.869270 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:55.869517 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:56.017308 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:56.017308 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:56.423424 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:56.573349 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:56.573870 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:56.744144 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:56.744411 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:56.988238 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:57.122723 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:57.124758 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:57.269229 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:57.269401 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:57.450007 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:57.589608 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:57.590129 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:57.701534 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:57.701911 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:57.896899 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:58.045807 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:58.045941 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:58.196711 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:58.196931 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:58.388002 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:58.521507 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:58.521827 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:58.679311 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:58.679869 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:59.295050 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:47:59.407079 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:47:59.407079 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:47:59.592011 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:47:59.592589 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:47:59.980575 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:00.130970 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:00.131238 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:00.279091 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:00.279238 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:00.421522 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:00.517658 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:00.517943 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:00.675197 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:00.675197 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:00.816166 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:00.950947 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:00.951358 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:01.081297 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:01.081673 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:01.196674 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:01.324395 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:01.324649 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:01.479002 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:01.479171 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:01.618605 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:01.744809 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:01.748650 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:01.888293 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:01.888554 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:02.036866 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:02.169320 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:02.169931 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:02.331118 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:02.331301 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:02.540603 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:02.683091 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:02.683201 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:02.792732 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:02.793135 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:02.988399 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:03.154000 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:03.154297 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:03.321579 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:03.321579 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:03.416674 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:03.491686 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:03.491686 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:03.580646 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:03.580897 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:03.587450 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:48:03.593694 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:48:03.593864 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:48:03.597798 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:48:03.597798 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:48:03.600669 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 21 15:48:03.600902 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1461043610, sha256file filename: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:48:03.692792 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:48:03.693191 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1461043610, metadata filename: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:48:03.730732 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "cefd6587-d307-41e5-9c94-3ccd62e93a0a",\n "backup_name": "tempest-VolumesBackupsTest-Backup-699357810",\n "created_at": "2025-11-21 20:47:46+00:00",\n "objects": [\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "82571f7121cb9475563edde615e4e4df",\n "offset": 0\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "a35891500449df17e893f84bd88472fb",\n "offset": 104857600\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "3750d5f1-2033-482d-a330-fc1c3f08b018",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"3750d5f1-2033-482d-a330-fc1c3f08b018\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"78aab0b8a676406dab9c030d6a4b429f\\", \\"project_id\\": \\"99dea2ee54874291a7a3284a115f81e4\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:47:44.000000\\", \\"launched_at\\": \\"2025-11-21T20:47:45.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1393104009\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 0\\", \\"provider_auth\\": \\"CHAP NAKNU4xYFLjMjZbV6BVE h4uoJMg2QvUjE8aK\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": true, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:47:43.000000\\", \\"updated_at\\": \\"2025-11-21T20:47:48.000000\\"}, \\"volume-glance-metadata\\": {\\"hw_rng_model\\": \\"virtio\\", \\"owner_specified.openstack.md5\\": \\"\\", \\"owner_specified.openstack.sha256\\": \\"\\", \\"owner_specified.openstack.object\\": \\"images/cirros-0.6.3-x86_64-disk\\", \\"os_glance_importing_to_stores\\": \\"\\", \\"os_glance_failed_import\\": \\"\\", \\"image_id\\": \\"dd592e5b-3302-490d-b4c0-d7eee3f65367\\", \\"image_name\\": \\"cirros-0.6.3-x86_64-disk\\", \\"checksum\\": \\"87617e24a5e30cb3b87fda8c0764838f\\", \\"container_format\\": \\"bare\\", \\"disk_format\\": \\"qcow2\\", \\"min_disk\\": \\"0\\", \\"min_ram\\": \\"0\\", \\"size\\": \\"21692416\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:48:03.738763 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] backup cefd6587-d307-41e5-9c94-3ccd62e93a0a finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:48:03.836273 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:03.865286 np0005531460.novalocal sudo[146662]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 21 15:48:04.199311 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.363s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.200939 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018', 'target_lun': 0, 'volume_id': '3750d5f1-2033-482d-a330-fc1c3f08b018', 'auth_method': 'CHAP', 'auth_username': 'NAKNU4xYFLjMjZbV6BVE', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405e114ca4236d04fafb184eaef5', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:04.201277 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:04.202015 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:04.204015 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.217017 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.217223 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcd995b4-9bae-409f-ba50-18ade64ffc4f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:04.217874 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:04.218239 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:48:04.218239 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:48:04.218239 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:48:04.218712 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:48:04.219833 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.230283 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.230548 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e12e86d-54ca-47af-a54f-0257cd4d9dda]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.231839 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.242941 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.243269 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[25c3f8dd-300d-4643-9290-9c530b9eab5c]: (4, ('tcp: [24] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.243900 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [24] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 (non-flash) Nov 21 15:48:04.243900 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:04.244255 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [24] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 (non-flash) Nov 21 15:48:04.244255 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:04.245404 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb83cd940>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018'): ({'sda'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:48:04.245868 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sda {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:48:04.246962 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.253635 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[648c75ab-22cf-467b-9c38-8d9a2a2142d5]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:48:04.256154 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:48:04.257919 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[648c75ab-22cf-467b-9c38-8d9a2a2142d5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.258613 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sda {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:48:04.259611 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.267428 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.267693 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1889dd0c-b60d-46ba-8476-d65fcc5a630b]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.268130 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:48:04.269020 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.321182 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.051s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.321353 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[eaf72d58-8f7e-45fd-abad-b864a2accfd1]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.322449 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:48:04.322449 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:48:04.322898 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:48:04.324518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.334518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.334811 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2591520c-9917-4967-8d8b-46393e6fb97a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.335413 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:04.336503 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.395667 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --logout" returned: 0 in 0.059s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.396394 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1aa56b00-d7e2-4d78-9d2e-9d75d5b7773a]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018, portal: 38.102.83.94,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.397019 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018, portal: 38.102.83.94,3260] Nov 21 15:48:04.397019 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018, portal: 38.102.83.94,3260] successful. Nov 21 15:48:04.397019 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:04.398090 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:04.408691 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:04.409115 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[35a82ded-59d1-43e6-a1ea-c25ff4938f3c]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:04.409718 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:04.412804 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.208s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:04.412804 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (209ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:07.948147 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-55429956-ecca-44c5-ba4e-4b32a8e66aa7 tempest-VolumesBackupsTest-633227990 None] Create backup finished. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a. Nov 21 15:48:09.290870 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Restore backup started, backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. Nov 21 15:48:09.301371 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:48:09.301628 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:48:09.302152 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:09.303920 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:09.311751 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:09.312133 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[29e4ef18-46a1-49d7-b324-e651d8a2294a]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f58152cd-c187-4dca-ac66-f11099801d90]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:09.314266 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:48:09.314949 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f58152cd-c187-4dca-ac66-f11099801d90]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:09.315517 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:48:09.315981 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:48:09.316867 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:09.324972 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:09.325220 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[04c91aa2-7cff-41bf-9bd6-e9fff025b610]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:09.326213 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:09.335598 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:09.335908 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[959d50c8-f10b-439a-bc44-829d902d421a]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:09.336983 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b411d814-89fe-426c-b52e-c216aa223801]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:09.337439 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:09.358639 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:09.358999 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:48:09.360938 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:48:09.362054 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194', 'fe80::fc16:3eff:febe:2819', 'fe80::fc16:3eff:fec9:cff8'] Nov 21 15:48:09.362309 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:48:09.362559 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:48:09.363120 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (60ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef0:4082', 'fe80::dc32:ff:fe93:194', 'fe80::fc16:3eff:febe:2819', 'fe80::fc16:3eff:fec9:cff8']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:09.477045 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-15f42853-72a0-43ee-aaf7-aacea4c792ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:48:13.882008 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:48:13.883686 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'target_lun': 0, 'volume_id': '7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'auth_method': 'CHAP', 'auth_username': 'XQUNVVy3FwkmdQziqTpJ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:13.884384 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:13.885347 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:13.885796 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'target_lun': 0, 'volume_id': '7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'auth_method': 'CHAP', 'auth_username': 'XQUNVVy3FwkmdQziqTpJ', 'auth_password': 'nPRwVN9PqAm7V4Bu', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:48:13.887476 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:13.894958 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a52547ae-21ce-4a02-95c0-0cb79762979c]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:48:13.896924 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:48:13.898914 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a52547ae-21ce-4a02-95c0-0cb79762979c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:13.900502 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:13.901655 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:13.902137 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:48:13.903368 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:13.914345 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:13.915592 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[edb3fed5-6220-4ba6-a155-99ae77113531]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:13.916091 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:48:13.916091 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:13.917177 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:13.928354 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:13.928702 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2482a32-f2b1-48e6-82a6-e186c950e5db]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:13.929234 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3] added Nov 21 15:48:13.929234 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:48:13.929234 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:48:13.929234 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:48:13.929234 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:13.929452 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:48:13.930150 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.028(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:48:13.930478 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:48:14.931758 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:48:14.933969 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:14.946291 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:14.946615 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[70395d84-6cf7-4912-a880-21d490ac1948]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:48:14.948553 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:14.954704 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:14.963902 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:14.964682 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f72eb10-2ecb-4c76-81ba-f4cb66054d86]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:14.965063 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:14.969566 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:14.984744 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:14.985075 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[00925992-20cb-4ed9-bc26-6c42c213d59d]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:14.985991 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:14.987159 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v XQUNVVy3FwkmdQziqTpJ {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.001169 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v XQUNVVy3FwkmdQziqTpJ" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.001382 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0b2b1c4-7fec-46fa-8bf0-a5a0385ea7b8]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.001955 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'XQUNVVy3FwkmdQziqTpJ'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:15.003429 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.019650 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.020697 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b59939de-912f-4a40-a2f8-04bbf4af982e]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.021235 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:15.022527 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.035196 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.036862 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ddc2d01-8e00-4f78-b96d-e2aa117eea40]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.037272 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 21 15:48:15.037272 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:15.037513 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 21 15:48:15.037513 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:15.037864 np0005531460.novalocal cinder-backup[107086]: WARNING os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 21 15:48:15.037864 np0005531460.novalocal cinder-backup[107086]: Nov 21 15:48:15.039799 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.059056 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --login" returned: 0 in 0.019s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.059595 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[44e60227-1c62-481f-94c4-a4dcb8608e4f]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.060182 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, portal: 38.102.83.94,3260] successful. Nov 21 15:48:15.060182 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:15.061113 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.075555 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.077299 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ec62ff2-ac38-4514-a050-d47ff31d681c]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.077648 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:15.078436 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.094453 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.016s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.094642 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fac351f1-fcbe-4a94-944c-b12a0f08ba2d]: (4, ('tcp: [25] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.095818 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [25] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 (non-flash) Nov 21 15:48:15.095818 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:15.095818 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [25] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 (non-flash) Nov 21 15:48:15.095818 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:15.095818 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.194s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:15.095818 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:48:15.096327 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] HCTL ('2', '-', '-', 0) found on session 25 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:48:15.096401 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:48:15.097201 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:15.123529 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.026s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:15.123710 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[60463c39-5500-496b-937c-2cc37c9aa3af]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:15.124550 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 25 and hctl ['2', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:48:16.126060 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 25 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:48:16.127120 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'target_lun': 0, 'volume_id': '7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'auth_method': 'CHAP', 'auth_username': 'XQUNVVy3FwkmdQziqTpJ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:48:16.128035 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.243s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:16.128579 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2244ms) {'type': 'block', 'scsi_wwn': '36001405a1b98c0fabfa4f98987454ad0', 'path': '/dev/sda'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:16.129070 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:16.169722 np0005531460.novalocal sudo[147121]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 21 15:48:16.486931 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.357s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:16.487324 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] starting restore of backup volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a container: tempest-volumesbackupstest-backup-container-1461043610, to new volume 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a. {{(pid=107086) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:783}} Nov 21 15:48:16.487465 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1461043610, metadata filename: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_metadata. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] _read_metadata finished. Metadata: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "backup_description": null, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "backup_id": "cefd6587-d307-41e5-9c94-3ccd62e93a0a", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "backup_name": "tempest-VolumesBackupsTest-Backup-699357810", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "created_at": "2025-11-21 20:47:46+00:00", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "objects": [ Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "82571f7121cb9475563edde615e4e4df", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 0 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 52428800 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "a35891500449df17e893f84bd88472fb", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 104857600 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 157286400 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 209715200 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 262144000 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 314572800 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 367001600 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 419430400 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 471859200 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 524288000 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 576716800 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 629145600 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 681574400 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 734003200 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 786432000 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 838860800 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 891289600 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 943718400 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 996147200 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021": { Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "length": 25165824, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "offset": 1048576000 Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: ], Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "parent_id": null, Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "version": "1.0.0", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_id": "3750d5f1-2033-482d-a330-fc1c3f08b018", Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"3750d5f1-2033-482d-a330-fc1c3f08b018\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"78aab0b8a676406dab9c030d6a4b429f\", \"project_id\": \"99dea2ee54874291a7a3284a115f81e4\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-21T20:47:44.000000\", \"launched_at\": \"2025-11-21T20:47:45.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-1393104009\", \"display_description\": null, \"provider_location\": \"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 0\", \"provider_auth\": \"CHAP NAKNU4xYFLjMjZbV6BVE h4uoJMg2QvUjE8aK\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"099441c3-a89d-49d9-8e0f-b86f1daa5de4\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": true, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"556adf35-c5a6-4979-a942-981586090f3a\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-21T20:47:43.000000\", \"updated_at\": \"2025-11-21T20:47:48.000000\"}, \"volume-glance-metadata\": {\"hw_rng_model\": \"virtio\", \"owner_specified.openstack.md5\": \"\", \"owner_specified.openstack.sha256\": \"\", \"owner_specified.openstack.object\": \"images/cirros-0.6.3-x86_64-disk\", \"os_glance_importing_to_stores\": \"\", \"os_glance_failed_import\": \"\", \"image_id\": \"dd592e5b-3302-490d-b4c0-d7eee3f65367\", \"image_name\": \"cirros-0.6.3-x86_64-disk\", \"checksum\": \"87617e24a5e30cb3b87fda8c0764838f\", \"container_format\": \"bare\", \"disk_format\": \"qcow2\", \"min_disk\": \"0\", \"min_ram\": \"0\", \"size\": \"21692416\"}}" Nov 21 15:48:16.527684 np0005531460.novalocal cinder-backup[107086]: }. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 21 15:48:16.531399 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Restoring backup version 1.0.0 {{(pid=107086) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:796}} Nov 21 15:48:16.531399 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1461043610, metadata filename: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_metadata. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] _read_metadata finished. Metadata: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "backup_description": null, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "backup_id": "cefd6587-d307-41e5-9c94-3ccd62e93a0a", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "backup_name": "tempest-VolumesBackupsTest-Backup-699357810", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "created_at": "2025-11-21 20:47:46+00:00", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "objects": [ Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "82571f7121cb9475563edde615e4e4df", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 0 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 52428800 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "a35891500449df17e893f84bd88472fb", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 104857600 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 157286400 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 209715200 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 262144000 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 314572800 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 367001600 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 419430400 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 471859200 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 524288000 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 576716800 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 629145600 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 681574400 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 734003200 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 786432000 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 838860800 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 891289600 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 943718400 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 996147200 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021": { Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "length": 25165824, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "offset": 1048576000 Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: ], Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "parent_id": null, Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "version": "1.0.0", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_id": "3750d5f1-2033-482d-a330-fc1c3f08b018", Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"3750d5f1-2033-482d-a330-fc1c3f08b018\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"78aab0b8a676406dab9c030d6a4b429f\", \"project_id\": \"99dea2ee54874291a7a3284a115f81e4\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-21T20:47:44.000000\", \"launched_at\": \"2025-11-21T20:47:45.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-1393104009\", \"display_description\": null, \"provider_location\": \"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 iqn.2010-10.org.openstack:volume-3750d5f1-2033-482d-a330-fc1c3f08b018 0\", \"provider_auth\": \"CHAP NAKNU4xYFLjMjZbV6BVE h4uoJMg2QvUjE8aK\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"099441c3-a89d-49d9-8e0f-b86f1daa5de4\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": true, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"556adf35-c5a6-4979-a942-981586090f3a\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-21T20:47:43.000000\", \"updated_at\": \"2025-11-21T20:47:48.000000\"}, \"volume-glance-metadata\": {\"hw_rng_model\": \"virtio\", \"owner_specified.openstack.md5\": \"\", \"owner_specified.openstack.sha256\": \"\", \"owner_specified.openstack.object\": \"images/cirros-0.6.3-x86_64-disk\", \"os_glance_importing_to_stores\": \"\", \"os_glance_failed_import\": \"\", \"image_id\": \"dd592e5b-3302-490d-b4c0-d7eee3f65367\", \"image_name\": \"cirros-0.6.3-x86_64-disk\", \"checksum\": \"87617e24a5e30cb3b87fda8c0764838f\", \"container_format\": \"bare\", \"disk_format\": \"qcow2\", \"min_disk\": \"0\", \"min_ram\": \"0\", \"size\": \"21692416\"}}" Nov 21 15:48:16.544643 np0005531460.novalocal cinder-backup[107086]: }. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 21 15:48:16.547313 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] v1 volume backup restore of cefd6587-d307-41e5-9c94-3ccd62e93a0a started. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:700}} Nov 21 15:48:16.547313 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] metadata_object_names = ['volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021']. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:707}} Nov 21 15:48:16.581224 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_metadata', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:48:16.585653 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:16.672963 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:17.150093 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:17.172731 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:17.429974 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:17.461204 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:17.716724 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:17.734011 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:17.936381 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:17.955693 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:18.136880 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:18.160438 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:18.358644 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:18.375147 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:18.576052 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:18.594063 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:18.805336 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:18.827052 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:19.055291 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:19.080651 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:19.293739 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:19.315807 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:19.560739 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:19.583333 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:19.811933 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:19.842342 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:20.097986 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:20.119648 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:20.330425 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:20.349160 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:20.529826 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:20.547993 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:20.769549 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:20.788672 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:20.971038 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:20.987153 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:21.195449 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:21.220819 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:21.493107 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:21.509660 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:21.724642 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, object name: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021, volume: 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:48:21.737974 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:48:21.823804 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] v1 volume backup restore of cefd6587-d307-41e5-9c94-3ccd62e93a0a finished. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:772}} Nov 21 15:48:21.824045 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Restoring volume base metadata {{(pid=107086) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 21 15:48:21.837525 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] No metadata of type 'volume-metadata' to restore {{(pid=107086) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 21 15:48:21.837625 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Restoring volume glance metadata {{(pid=107086) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 21 15:48:21.936691 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] restore cefd6587-d307-41e5-9c94-3ccd62e93a0a to 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 finished. {{(pid=107086) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:837}} Nov 21 15:48:21.942895 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:21.971827 np0005531460.novalocal sudo[147209]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 21 15:48:22.303825 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.360s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.305385 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'target_lun': 0, 'volume_id': '7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 'auth_method': 'CHAP', 'auth_username': 'XQUNVVy3FwkmdQziqTpJ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405a1b98c0fabfa4f98987454ad0', 'path': '/dev/sda'}), 'kwargs': {'force': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:22.305715 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:22.306434 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:22.308179 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.320048 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.320315 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fcd0a4c-bc08-481a-b303-f5fa4531f10d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:22.320931 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:22.321228 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:48:22.321228 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:48:22.321228 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:48:22.321737 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:48:22.322691 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.330810 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.331046 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[18337668-1af0-4b02-9a6e-ee2d94f352a1]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.332539 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.344078 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.344325 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9af8199-879c-4a90-bf43-c13fb30f70c3]: (4, ('tcp: [25] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.344754 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [25] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 (non-flash) Nov 21 15:48:22.344754 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:22.345085 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [25] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 (non-flash) Nov 21 15:48:22.345085 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:22.345985 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb83f3ba0>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3'): ({'sda'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:48:22.346366 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sda {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:48:22.347298 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.352829 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6239a772-8811-4311-90c7-6166bfafc91c]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:48:22.354631 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:48:22.355437 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6239a772-8811-4311-90c7-6166bfafc91c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.356056 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sda {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:48:22.356896 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.364582 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.364877 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c12926f4-e974-403b-b618-9345db64d34f]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.365271 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:48:22.366034 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.404052 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.038s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.404298 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ade0f9e-3ef9-4289-b172-6d975ef5d0e6]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.404749 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:48:22.405058 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:48:22.405451 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:48:22.406520 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.418116 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.418273 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e47bb6ff-8837-4353-af71-aacf237c2c27]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.418698 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:22.419554 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.451971 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --logout" returned: 0 in 0.031s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.452160 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce060a6b-f66c-4338-a5d1-5182a0c09cf4]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, portal: 38.102.83.94,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.452632 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, portal: 38.102.83.94,3260] Nov 21 15:48:22.452632 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3, portal: 38.102.83.94,3260] successful. Nov 21 15:48:22.452632 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:22.454737 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:22.467437 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:22.467691 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a97cf6d9-9094-40b9-8c4d-9ac7606139b4]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:22.468260 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:22.468758 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.162s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:22.469098 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (163ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:25.994841 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.manager [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Encryption key ID for volume 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3 already matches encryption key ID in backup cefd6587-d307-41e5-9c94-3ccd62e93a0a. {{(pid=107086) _run_restore /opt/stack/cinder/cinder/backup/manager.py:874}} Nov 21 15:48:26.022636 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-310ff936-2ba4-4732-9a63-4a050da9f123 tempest-VolumesBackupsTest-633227990 None] Finished restoring backup cefd6587-d307-41e5-9c94-3ccd62e93a0a to volume 7f0fdd3f-26c8-4784-b8aa-cd1d47edffc3. Nov 21 15:48:29.195393 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] Delete backup started, backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a. Nov 21 15:48:29.197393 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:48:29.197665 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:48:29.197992 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] delete started, backup: cefd6587-d307-41e5-9c94-3ccd62e93a0a, container: tempest-volumesbackupstest-backup-container-1461043610, prefix: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:48:29.249940 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_metadata', 'volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:48:29.277725 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00001 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.300793 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00002 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.324151 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00003 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.349696 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00004 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.377700 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00005 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.407144 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00006 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.433724 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00007 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.463929 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00008 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.493623 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00009 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.526286 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00010 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.556390 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00011 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.586830 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00012 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.615873 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00013 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.645350 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00014 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.677458 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00015 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.710819 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00016 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.741669 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00017 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.775692 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00018 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.811182 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00019 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.841216 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00020 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.866721 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a-00021 in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.901132 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_metadata in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.935021 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] deleted object: volume_3750d5f1-2033-482d-a330-fc1c3f08b018/20251121204752/az_nova_backup_cefd6587-d307-41e5-9c94-3ccd62e93a0a_sha256file in container: tempest-volumesbackupstest-backup-container-1461043610. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:48:29.935021 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] delete cefd6587-d307-41e5-9c94-3ccd62e93a0a finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:48:29.943020 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:48:29.976331 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] Created reservations ['7f684042-df77-4342-bde1-fbf59a383c4c', '39f78799-938e-49c7-81ce-b2a833a5c115'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:48:30.002804 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1251b510-4fe1-49e8-88af-07db6650087c tempest-VolumesBackupsTest-633227990 None] Delete backup finished, backup cefd6587-d307-41e5-9c94-3ccd62e93a0a deleted. Nov 21 15:48:34.087401 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Create backup started, backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b volume: fa8c80d2-e0e4-4acd-9fc6-90e671b33272. Nov 21 15:48:34.092204 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1179280645',created_at=2025-11-21T20:48:34Z,data_timestamp=2025-11-21T20:48:34Z,deleted=False,deleted_at=None,display_description='tempest-backup-description-1901327566',display_name='tempest-VolumesBackupsTest-Backup-503845795',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=981e0ddf-41dc-4826-8d26-7d7fbac9fe2b,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='99dea2ee54874291a7a3284a115f81e4',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='78aab0b8a676406dab9c030d6a4b429f',volume_id=fa8c80d2-e0e4-4acd-9fc6-90e671b33272) Nov 21 15:48:34.129678 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:48:34.130032 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:48:34.130525 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:34.131960 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:34.142602 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:34.143805 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4d79d9f-d368-4e93-8bb8-fae408cf7456]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[5ed56caf-c4e1-4cbc-8c1f-adbf62a28b88]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:34.146191 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:48:34.147434 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ed56caf-c4e1-4cbc-8c1f-adbf62a28b88]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:34.148169 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:48:34.148654 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:48:34.149879 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:34.162653 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:34.162889 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ae8bc01-db7a-451f-beb8-4c583a7b2025]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:34.164563 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:34.179462 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:34.179701 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[dae26f63-a433-41c4-8a6f-0038a41a9030]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:34.180952 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[34300c85-8cd9-4320-8ec2-3b68e41f8aeb]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:34.181593 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:34.200319 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:34.200700 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:48:34.202834 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:48:34.203741 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944'] Nov 21 15:48:34.204073 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:48:34.204329 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:48:34.204899 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (73ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:37.715683 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:48:37.717034 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'target_lun': 0, 'volume_id': 'fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'auth_method': 'CHAP', 'auth_username': 'FL26zR2Qg7PKU3nVAXzM', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:37.717366 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:37.718232 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:37.718582 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'target_lun': 0, 'volume_id': 'fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'auth_method': 'CHAP', 'auth_username': 'FL26zR2Qg7PKU3nVAXzM', 'auth_password': 'DXnPV9fJUgPAPzBk', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:48:37.719938 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:37.730179 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f01c64a1-ba97-4930-a783-fe16b7c7214d]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:48:37.732827 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:48:37.734519 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f01c64a1-ba97-4930-a783-fe16b7c7214d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:37.736122 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:37.737251 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:37.737675 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:48:37.739085 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:37.750314 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:37.750562 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9db0104e-a368-4fc9-8c17-723deb734461]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:37.751049 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:48:37.751049 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:37.752011 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:37.764052 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:37.764323 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[55f81662-5a22-47a3-9216-8d678fdc5189]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:37.764849 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272] added Nov 21 15:48:37.764849 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:48:37.764849 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:48:37.764849 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:48:37.764849 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:37.765133 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:48:37.766543 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.028(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:48:37.767086 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:48:38.769167 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:48:38.771813 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.784176 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.784559 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc86e557-a1af-425c-9c5f-7668d4f98021]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:48:38.786401 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.789178 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.799378 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.799824 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8366378e-243b-4333-b7d0-9a89f7c24bda]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.800257 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.801243 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.810287 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.810596 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[61c8f5a5-00cf-4818-a31f-dbcd56dff076]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.811038 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.811991 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v FL26zR2Qg7PKU3nVAXzM {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.826161 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v FL26zR2Qg7PKU3nVAXzM" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.826334 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[dffb17a6-c3b1-486c-8e37-d7a4790c8c9d]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.826716 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'FL26zR2Qg7PKU3nVAXzM'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.827811 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.843354 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.843684 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[afecbd69-f9cd-47d2-8b15-9fb33bd04b09]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.844256 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.845243 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.859259 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.859510 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e231eaae-d268-4afa-8607-604e9c4543b4]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.859980 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 21 15:48:38.859980 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:38.860236 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 21 15:48:38.860236 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:38.860502 np0005531460.novalocal cinder-backup[107086]: WARNING os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 21 15:48:38.860502 np0005531460.novalocal cinder-backup[107086]: Nov 21 15:48:38.861472 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.882669 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --login" returned: 0 in 0.021s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.882669 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[160e1f78-d1e7-47e1-8dba-788b96f0f53f]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.882865 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272, portal: 38.102.83.94,3260] successful. Nov 21 15:48:38.882865 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.885306 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.937269 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.052s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.937752 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cb36dd1-27e8-4508-87b2-1eb5a8d07168]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.938136 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:38.940610 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.950509 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.950862 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec4f8f76-89c0-4a53-99fe-dfaa3de28fed]: (4, ('tcp: [26] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.951159 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [26] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 (non-flash) Nov 21 15:48:38.951159 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:38.951334 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [26] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 (non-flash) Nov 21 15:48:38.951334 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:38.951743 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.215s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:38.951948 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:48:38.952409 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] HCTL ('2', '-', '-', 0) found on session 26 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:48:38.952586 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:48:38.953504 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:38.985735 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.030s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:38.985915 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[60ac02c2-bfd5-4c9b-b04d-a053cf556c0d]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:38.986420 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 26 and hctl ['2', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:48:39.988165 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 26 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:48:39.988733 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'target_lun': 0, 'volume_id': 'fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'auth_method': 'CHAP', 'auth_username': 'FL26zR2Qg7PKU3nVAXzM', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:48:39.989714 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.272s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:39.989999 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2272ms) {'type': 'block', 'scsi_wwn': '36001405680782e3105d4f8388a22e053', 'path': '/dev/sda'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:39.990640 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:40.018833 np0005531460.novalocal sudo[147564]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 21 15:48:40.482119 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.490s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:40.488501 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:48:40.491488 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=98,cinder:UPDATE=18,cinder:INSERT=17 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:48:40.502101 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1179280645,backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:48:40.585099 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] generate_object_name_prefix: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:48:40.591403 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] starting backup of volume: fa8c80d2-e0e4-4acd-9fc6-90e671b33272, volume size: 1073741824, object names prefix volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:48:40.967366 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:41.117491 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:41.117898 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:41.276836 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:41.277058 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:41.467821 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:41.602764 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:41.603002 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:41.739286 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:41.739562 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:41.907758 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:42.013251 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:42.013251 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:42.165429 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:42.165429 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:42.283035 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:42.383234 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:42.383353 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:42.526834 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:42.527070 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:42.672698 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:42.807685 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:42.807925 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:42.942578 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:42.942923 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:43.307264 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:43.441902 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:43.442006 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:43.592885 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:43.593126 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:43.749197 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:43.869257 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:43.869353 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:43.986228 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:43.986577 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:44.200150 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:44.334085 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:44.334517 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:44.458910 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:44.459120 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:44.948599 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:45.116397 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:45.116693 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:45.257220 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:45.257395 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:45.646641 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:45.800222 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:45.800222 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:45.979025 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:45.979988 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:46.204838 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:46.340414 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:46.340734 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:46.476579 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:46.476864 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:46.497511 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:48:46.497511 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:48:46.865204 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:46.999862 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:46.999976 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:47.147424 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:47.147868 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:47.394121 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:47.507834 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:47.508055 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:47.658902 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:47.659195 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:47.830157 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:47.965785 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:47.966278 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:48.103688 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:48.103970 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:48.293277 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:48.448225 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:48.448487 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:48.563799 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:48.564110 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:48.698353 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:48.875817 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:48.875993 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:49.028794 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:49.029395 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:49.200354 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:49.335507 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:49.335616 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:49.550915 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:49.551212 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:49.736503 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:49.859984 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:49.860258 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:50.009102 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:50.009404 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:50.185073 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:50.337691 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:50.338150 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:50.476816 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:50.477183 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:50.707882 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:50.818314 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:50.818497 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:50.969670 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:50.969830 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:51.066335 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:48:51.174344 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:48:51.174522 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:48:51.266955 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:48:51.267303 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:48:51.274990 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:48:51.287476 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:48:51.287859 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:48:51.293581 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:121}} Nov 21 15:48:51.293919 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:48:51.298617 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:48:51.299081 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1179280645, sha256file filename: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:48:51.396894 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:48:51.396894 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1179280645, metadata filename: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:48:51.434886 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] _write_metadata finished. Metadata: b'{\n "backup_description": "tempest-backup-description-1901327566",\n "backup_id": "981e0ddf-41dc-4826-8d26-7d7fbac9fe2b",\n "backup_name": "tempest-VolumesBackupsTest-Backup-503845795",\n "created_at": "2025-11-21 20:48:34+00:00",\n "objects": [\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "fa8c80d2-e0e4-4acd-9fc6-90e671b33272",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"fa8c80d2-e0e4-4acd-9fc6-90e671b33272\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"78aab0b8a676406dab9c030d6a4b429f\\", \\"project_id\\": \\"99dea2ee54874291a7a3284a115f81e4\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:48:32.000000\\", \\"launched_at\\": \\"2025-11-21T20:48:33.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-2124873287\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 0\\", \\"provider_auth\\": \\"CHAP FL26zR2Qg7PKU3nVAXzM DXnPV9fJUgPAPzBk\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:48:32.000000\\", \\"updated_at\\": \\"2025-11-21T20:48:36.000000\\"}, \\"volume-metadata\\": {\\"vol-meta1\\": \\"value1\\", \\"vol-meta2\\": \\"value2\\", \\"vol-meta3\\": \\"value3\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:48:51.440290 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] backup 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:48:51.522923 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.551813 np0005531460.novalocal sudo[147580]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 21 15:48:51.879391 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.356s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.880268 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'target_lun': 0, 'volume_id': 'fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 'auth_method': 'CHAP', 'auth_username': 'FL26zR2Qg7PKU3nVAXzM', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405680782e3105d4f8388a22e053', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:51.880454 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:48:51.880962 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.000s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:48:51.882206 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.891805 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.892104 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bd4c944-3b1f-475f-bdfb-fdae8e1d72e9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:48:51.892318 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:51.892591 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:48:51.892591 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:48:51.892591 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:48:51.892716 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:48:51.893283 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.900951 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.901184 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[23fe1809-b2fa-4050-a7ee-ab4863f0aad6]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.901666 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.910441 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.910812 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7cc4586b-0f79-4e7a-81c9-5088947d5e71]: (4, ('tcp: [26] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.911108 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [26] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 (non-flash) Nov 21 15:48:51.911108 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:48:51.911229 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [26] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 (non-flash) Nov 21 15:48:51.911229 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:48:51.911698 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddd67faf20>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272'): ({'sda'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:48:51.911919 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sda {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:48:51.913720 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.917195 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[58c2c8c3-4d3d-4f99-a8bf-22a99ff86f60]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:48:51.918325 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:48:51.919129 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[58c2c8c3-4d3d-4f99-a8bf-22a99ff86f60]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.920035 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sda {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:48:51.920950 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.925829 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.005s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.925999 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[993fee69-1c0c-4265-a523-7e0a087d2c9c]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.926310 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:48:51.927037 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.979887 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.052s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.980645 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[72f2484d-8645-4196-8450-e63800c44279]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.981200 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:48:51.981490 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:48:51.981952 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:48:51.983172 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:51.994856 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:51.995238 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e97d059a-865a-4452-bb6c-69c28fb32368]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:51.995583 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:51.996705 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:52.028866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --logout" returned: 0 in 0.032s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:52.029200 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7648026c-4754-4226-a90a-01c2fbca7892]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272, portal: 38.102.83.94,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:52.029655 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272, portal: 38.102.83.94,3260] Nov 21 15:48:52.029655 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272, portal: 38.102.83.94,3260] successful. Nov 21 15:48:52.029655 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:52.030707 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:52.045151 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:52.045405 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a9da5d0-43c4-4335-8e86-0b7e764cef2a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:52.045945 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:48:52.046263 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.165s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:48:52.046534 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (165ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:48:55.579391 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-3878dfda-6685-4313-b90d-71b60b7f5d0a tempest-VolumesBackupsTest-633227990 None] Create backup finished. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b. Nov 21 15:48:57.158806 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Restore backup started, backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. Nov 21 15:48:57.169991 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:48:57.170270 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:48:57.170984 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:48:57.172973 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:57.186688 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:57.186688 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[39913042-dbbb-43ce-b823-69a7bc169d9e]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[9abd986e-3f66-4ff4-bc34-b08affa131c1]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:48:57.188804 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:48:57.189320 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9abd986e-3f66-4ff4-bc34-b08affa131c1]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:57.190034 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:48:57.190491 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:48:57.193974 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:57.202144 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:57.202486 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[67158de5-e1d0-4e8c-992d-f91cd29af39c]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:57.203438 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:57.213271 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:48:57.213616 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f218b3f3-bdff-42d8-a8f1-1722e715dd45]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:57.214518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[34871698-efbb-4674-b1ee-894141260e23]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:48:57.215005 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:48:57.230212 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:48:57.230212 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:48:57.230212 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:48:57.231029 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944'] Nov 21 15:48:57.231029 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:48:57.231169 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:48:57.231525 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (60ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:49:01.264249 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:49:01.265576 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'target_lun': 0, 'volume_id': '501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'auth_method': 'CHAP', 'auth_username': 'omB3T6TnGMb8hNkepHD2', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:49:01.265916 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:49:01.266568 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:49:01.266884 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'target_lun': 0, 'volume_id': '501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'auth_method': 'CHAP', 'auth_username': 'omB3T6TnGMb8hNkepHD2', 'auth_password': '3pNcJhp5JEVs5Lyz', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:49:01.268313 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:01.276556 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[59ce3417-690b-4df8-b8bd-7f8a6af5c31f]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:49:01.278548 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:49:01.280295 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[59ce3417-690b-4df8-b8bd-7f8a6af5c31f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:01.281948 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:49:01.282364 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:49:01.282653 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:49:01.283531 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:01.295336 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260" returned: 21 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:01.295658 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a092ceb-e555-458b-9ca1-3b60ab763818]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:01.296030 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:49:01.296030 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:01.296933 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:01.311100 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:01.311488 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5aa9bb8-ea1b-48ae-8a31-6c339be03086]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:01.311774 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3] added Nov 21 15:49:01.311774 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:49:01.311774 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:49:01.311774 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:49:01.311774 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:01.311990 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:49:01.312434 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.030(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:49:01.312653 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:49:02.314476 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:49:02.315995 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.326859 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.327248 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9180d96d-fe49-4664-a33e-02a5656f73f0]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:49:02.329005 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.331166 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.343030 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.343287 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[07872974-b6f4-4556-83c4-9e3deb7718b0]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.343709 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.344665 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.356520 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.357071 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d938a5d2-f8ea-43e8-9786-9d007113a84c]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.357480 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.358383 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v omB3T6TnGMb8hNkepHD2 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.369529 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v omB3T6TnGMb8hNkepHD2" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.369929 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1009c1cc-fb38-4011-96c1-463abec2319b]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.370351 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'omB3T6TnGMb8hNkepHD2'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.371450 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.382319 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.382484 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fbe6858-c94c-4c0b-a83e-d66360319827]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.383060 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.383871 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.395354 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.395564 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e20883b4-8d47-4dd0-b358-4c08f77c0bc8]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.395997 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 21 15:49:02.395997 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:49:02.396260 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 21 15:49:02.396260 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:49:02.396524 np0005531460.novalocal cinder-backup[107086]: WARNING os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 21 15:49:02.396524 np0005531460.novalocal cinder-backup[107086]: Nov 21 15:49:02.397431 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.415568 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --login" returned: 0 in 0.018s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.415930 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[468ef57f-9765-4013-916f-f0c5521f7580]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.416352 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3, portal: 38.102.83.94,3260] successful. Nov 21 15:49:02.416352 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.418674 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.429663 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.431591 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb31232a-5f33-49a3-b217-0b14cf14eb17]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.431591 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:02.432298 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.444962 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.445384 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3bb88129-872a-4347-9874-429dd0390674]: (4, ('tcp: [27] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.446029 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [27] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 (non-flash) Nov 21 15:49:02.446029 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:49:02.446442 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [27] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 (non-flash) Nov 21 15:49:02.446442 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:49:02.447150 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.165s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:49:02.447620 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:49:02.448562 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] HCTL ('2', '-', '-', 0) found on session 27 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:49:02.448997 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:49:02.450107 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:02.473538 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.023s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:02.474390 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[74dc1869-ada6-4c92-947c-463654b73496]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:02.475215 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 27 and hctl ['2', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:49:03.477694 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 27 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:49:03.478692 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'target_lun': 0, 'volume_id': '501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'auth_method': 'CHAP', 'auth_username': 'omB3T6TnGMb8hNkepHD2', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:49:03.479461 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.213s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:49:03.479814 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2213ms) {'type': 'block', 'scsi_wwn': '36001405a93363ce8d6a4651a54dfdd5e', 'path': '/dev/sda'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:49:03.480437 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:03.512169 np0005531460.novalocal sudo[147768]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 21 15:49:03.900243 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.419s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:03.901508 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] starting restore of backup volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b container: tempest-volumesbackupstest-backup-container-1179280645, to new volume 501a1a66-aaab-42fc-8e4e-f292ce125cb3, backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b. {{(pid=107086) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:783}} Nov 21 15:49:03.901508 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1179280645, metadata filename: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_metadata. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] _read_metadata finished. Metadata: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "backup_description": "tempest-backup-description-1901327566", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "backup_id": "981e0ddf-41dc-4826-8d26-7d7fbac9fe2b", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "backup_name": "tempest-VolumesBackupsTest-Backup-503845795", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "created_at": "2025-11-21 20:48:34+00:00", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "objects": [ Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 0 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 52428800 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 104857600 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 157286400 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 209715200 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 262144000 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 314572800 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 367001600 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 419430400 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 471859200 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 524288000 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 576716800 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 629145600 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 681574400 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 734003200 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 786432000 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 838860800 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 891289600 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 943718400 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 996147200 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021": { Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "length": 25165824, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "offset": 1048576000 Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: ], Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "parent_id": null, Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "version": "1.0.0", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_id": "fa8c80d2-e0e4-4acd-9fc6-90e671b33272", Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"fa8c80d2-e0e4-4acd-9fc6-90e671b33272\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"78aab0b8a676406dab9c030d6a4b429f\", \"project_id\": \"99dea2ee54874291a7a3284a115f81e4\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-21T20:48:32.000000\", \"launched_at\": \"2025-11-21T20:48:33.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-2124873287\", \"display_description\": null, \"provider_location\": \"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 0\", \"provider_auth\": \"CHAP FL26zR2Qg7PKU3nVAXzM DXnPV9fJUgPAPzBk\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"099441c3-a89d-49d9-8e0f-b86f1daa5de4\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": false, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"556adf35-c5a6-4979-a942-981586090f3a\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-21T20:48:32.000000\", \"updated_at\": \"2025-11-21T20:48:36.000000\"}, \"volume-metadata\": {\"vol-meta1\": \"value1\", \"vol-meta2\": \"value2\", \"vol-meta3\": \"value3\"}}" Nov 21 15:49:03.928151 np0005531460.novalocal cinder-backup[107086]: }. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 21 15:49:03.935181 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Restoring backup version 1.0.0 {{(pid=107086) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:796}} Nov 21 15:49:03.935181 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1179280645, metadata filename: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_metadata. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] _read_metadata finished. Metadata: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "backup_description": "tempest-backup-description-1901327566", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "backup_id": "981e0ddf-41dc-4826-8d26-7d7fbac9fe2b", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "backup_name": "tempest-VolumesBackupsTest-Backup-503845795", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "created_at": "2025-11-21 20:48:34+00:00", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "objects": [ Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 0 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 52428800 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 104857600 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 157286400 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 209715200 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 262144000 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 314572800 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 367001600 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 419430400 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 471859200 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 524288000 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 576716800 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 629145600 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 681574400 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 734003200 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 786432000 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 838860800 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 891289600 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 943718400 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 52428800, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 996147200 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021": { Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "compression": "zlib", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "length": 25165824, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "offset": 1048576000 Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: } Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: ], Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "parent_id": null, Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "version": "1.0.0", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_id": "fa8c80d2-e0e4-4acd-9fc6-90e671b33272", Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"fa8c80d2-e0e4-4acd-9fc6-90e671b33272\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"78aab0b8a676406dab9c030d6a4b429f\", \"project_id\": \"99dea2ee54874291a7a3284a115f81e4\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-21T20:48:32.000000\", \"launched_at\": \"2025-11-21T20:48:33.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-2124873287\", \"display_description\": null, \"provider_location\": \"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 iqn.2010-10.org.openstack:volume-fa8c80d2-e0e4-4acd-9fc6-90e671b33272 0\", \"provider_auth\": \"CHAP FL26zR2Qg7PKU3nVAXzM DXnPV9fJUgPAPzBk\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"099441c3-a89d-49d9-8e0f-b86f1daa5de4\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": false, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"556adf35-c5a6-4979-a942-981586090f3a\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-21T20:48:32.000000\", \"updated_at\": \"2025-11-21T20:48:36.000000\"}, \"volume-metadata\": {\"vol-meta1\": \"value1\", \"vol-meta2\": \"value2\", \"vol-meta3\": \"value3\"}}" Nov 21 15:49:03.946733 np0005531460.novalocal cinder-backup[107086]: }. {{(pid=107086) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 21 15:49:03.952659 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] v1 volume backup restore of 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b started. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:700}} Nov 21 15:49:03.952659 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] metadata_object_names = ['volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021']. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:707}} Nov 21 15:49:04.013638 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_metadata', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:49:04.019493 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:49:04.026948 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:04.049241 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:04.300316 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:04.319486 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:04.560210 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:04.580261 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:04.777028 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:04.796225 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:05.007338 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:05.027289 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:05.218462 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:05.238956 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:05.451930 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:05.472484 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:05.665520 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:05.684762 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:05.891890 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:05.914911 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:06.137843 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:06.156288 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:06.332719 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:06.351288 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:06.559656 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:06.578691 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:06.825104 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:06.844992 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:07.029855 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:07.046839 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:07.250625 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:07.268689 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:07.469325 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:07.495228 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:07.728733 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:07.746234 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:07.939242 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:07.956476 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:08.136141 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:08.153834 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:08.364857 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:08.380242 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:08.558640 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restoring object. backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, object name: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021, volume: 501a1a66-aaab-42fc-8e4e-f292ce125cb3. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 21 15:49:08.574278 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] decompressing data using zlib algorithm {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 21 15:49:08.667807 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] v1 volume backup restore of 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b finished. {{(pid=107086) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:772}} Nov 21 15:49:08.668012 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Restoring volume base metadata {{(pid=107086) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 21 15:49:08.689452 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Restoring volume metadata {{(pid=107086) _restore_vol_meta /opt/stack/cinder/cinder/backup/driver.py:249}} Nov 21 15:49:08.698371 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=107086) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 21 15:49:08.698507 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] restore 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b to 501a1a66-aaab-42fc-8e4e-f292ce125cb3 finished. {{(pid=107086) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:837}} Nov 21 15:49:08.699338 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:08.727200 np0005531460.novalocal sudo[147781]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 21 15:49:09.034602 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.335s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.035419 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'target_lun': 0, 'volume_id': '501a1a66-aaab-42fc-8e4e-f292ce125cb3', 'auth_method': 'CHAP', 'auth_username': 'omB3T6TnGMb8hNkepHD2', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405a93363ce8d6a4651a54dfdd5e', 'path': '/dev/sda'}), 'kwargs': {'force': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:49:09.035590 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:49:09.035980 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.000s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:49:09.037179 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.046982 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.047365 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa9622aa-1314-48d0-baa9-24e6b6a12414]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:49:09.047632 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:49:09.047965 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:49:09.047965 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:49:09.047965 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:49:09.048076 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:49:09.048986 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.057862 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.058423 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef8e3198-435d-49b6-a2df-43b6b92a97c4]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.059276 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.069348 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.069475 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9518bed9-2431-4cf5-99e3-f65794fb65df]: (4, ('tcp: [27] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.069666 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [27] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 (non-flash) Nov 21 15:49:09.069666 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:49:09.069805 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [27] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 (non-flash) Nov 21 15:49:09.069805 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:49:09.070263 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb834a8e0>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3'): ({'sda'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:49:09.070448 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sda {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:49:09.071129 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.078570 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e28ba532-8928-4718-b15d-b6ee1115a88e]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:49:09.080741 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:49:09.082418 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e28ba532-8928-4718-b15d-b6ee1115a88e]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.082715 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sda {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:49:09.083282 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.093172 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.093496 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e8dc887-a3e3-4dbc-9e5b-581e01a5ce8f]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.093668 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:49:09.094366 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.142230 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.047s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.142517 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[42e0dc7e-a97e-4f67-9697-423485b2d619]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.143077 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:49:09.143370 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:49:09.143775 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:49:09.145003 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.157483 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.158962 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6d01075-e472-45ec-b447-0202c827a05a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.158962 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:09.159728 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.203945 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --logout" returned: 0 in 0.044s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.204168 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[90ffdc56-4538-41cd-a41a-72eccc904fec]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3, portal: 38.102.83.94,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.204696 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3, portal: 38.102.83.94,3260] Nov 21 15:49:09.204696 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3, portal: 38.102.83.94,3260] successful. Nov 21 15:49:09.204696 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:09.205750 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:49:09.217614 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-501a1a66-aaab-42fc-8e4e-f292ce125cb3 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:49:09.217878 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe98e65f-6771-41b5-8bde-c242140ecb72]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:49:09.218429 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:49:09.218708 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.183s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:49:09.219126 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (183ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:49:12.742856 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.manager [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Encryption key ID for volume 501a1a66-aaab-42fc-8e4e-f292ce125cb3 already matches encryption key ID in backup 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b. {{(pid=107086) _run_restore /opt/stack/cinder/cinder/backup/manager.py:874}} Nov 21 15:49:12.761174 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-f6b07fe0-2ae2-4288-b3a8-1f27e1ef6006 tempest-VolumesBackupsTest-633227990 None] Finished restoring backup 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b to volume 501a1a66-aaab-42fc-8e4e-f292ce125cb3. Nov 21 15:49:16.001349 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] Delete backup started, backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b. Nov 21 15:49:16.002168 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:49:16.002290 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:49:16.002435 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] delete started, backup: 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b, container: tempest-volumesbackupstest-backup-container-1179280645, prefix: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:49:16.047902 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_metadata', 'volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:49:16.078251 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00001 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.107887 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00002 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.138176 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00003 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.163655 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00004 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.189570 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00005 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.211487 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00006 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.240310 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00007 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.263814 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00008 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.294303 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00009 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.321459 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00010 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.344916 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00011 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.368473 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00012 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.392644 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00013 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.464015 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00014 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.511358 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00015 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.539975 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00016 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.567806 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00017 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.591656 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00018 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.618038 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00019 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.644829 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00020 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.678454 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b-00021 in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.705008 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_metadata in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.733343 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_fa8c80d2-e0e4-4acd-9fc6-90e671b33272/20251121204840/az_nova_backup_981e0ddf-41dc-4826-8d26-7d7fbac9fe2b_sha256file in container: tempest-volumesbackupstest-backup-container-1179280645. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:49:16.733721 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] delete 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:49:16.738737 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:49:16.765432 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] Created reservations ['985c204e-00c7-4bae-8d27-f2d556d0aebb', '6b23c3ac-0111-4f33-8b52-9ec4114cbf10'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:49:16.785989 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-14047574-4308-41ce-8763-b9129861d245 tempest-VolumesBackupsTest-633227990 None] Delete backup finished, backup 981e0ddf-41dc-4826-8d26-7d7fbac9fe2b deleted. Nov 21 15:49:26.785978 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:UPDATE=14,cinder:SELECT=65,cinder:INSERT=4 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:49:46.495475 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:49:46.496193 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:50:16.069582 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Create backup started, backup: f2011b7c-6706-4260-a22e-269dff7e0233 volume: 6ef3b23a-f7b5-420d-86b8-2fb15f829882. Nov 21 15:50:16.073375 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1902449308',created_at=2025-11-21T20:50:16Z,data_timestamp=2025-11-21T20:50:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-81611907',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=f2011b7c-6706-4260-a22e-269dff7e0233,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='99dea2ee54874291a7a3284a115f81e4',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='78aab0b8a676406dab9c030d6a4b429f',volume_id=6ef3b23a-f7b5-420d-86b8-2fb15f829882) Nov 21 15:50:17.484426 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:50:17.484426 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:50:17.484955 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:50:17.486250 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:17.495868 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:17.496600 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb831939-772d-4118-bf76-f4efe9ea66dc]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a73d6e2b-5f77-4360-a4fb-faead8869440]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:17.499350 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:50:17.499864 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a73d6e2b-5f77-4360-a4fb-faead8869440]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:17.500355 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:50:17.500838 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:50:17.501864 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:17.512788 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:17.512867 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6209707f-09c9-4767-8e06-f6dfef739339]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:17.513937 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:17.557447 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.043s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:17.557814 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[84b6afde-d066-460a-8ed7-9c678423bd82]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:17.559261 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[551fba4d-1b25-4d28-8866-60a5b3718cdc]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:17.559866 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:17.576225 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:50:17.576653 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:50:17.578689 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:50:17.579868 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef8:dc6', 'fe80::34:b9ff:fe8e:4a20'] Nov 21 15:50:17.580161 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:50:17.580438 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:50:17.581038 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (95ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef8:dc6', 'fe80::34:b9ff:fe8e:4a20']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:50:21.091721 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:50:21.092889 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'target_lun': 0, 'volume_id': 'ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'auth_method': 'CHAP', 'auth_username': 'oiEx9AfBKxyfq8m4B4wJ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:50:21.093183 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:50:21.094346 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:50:21.094346 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'target_lun': 0, 'volume_id': 'ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'auth_method': 'CHAP', 'auth_username': 'oiEx9AfBKxyfq8m4B4wJ', 'auth_password': 'NMR6WPba9iYXyTTt', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:50:21.095735 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:21.101958 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b31df049-4791-405b-bf15-b8d328d23009]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:50:21.104290 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:50:21.106660 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b31df049-4791-405b-bf15-b8d328d23009]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:21.106660 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:50:21.107854 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:50:21.107854 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:50:21.108743 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:21.117686 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260" returned: 21 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:21.117984 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[da30a3e9-a48e-408e-a94d-5c1187ec143e]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:21.118389 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:50:21.118389 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:21.119312 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:21.128636 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:21.129125 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[32859cda-c4f9-4260-8c68-bac741b9a73d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:21.129548 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc] added Nov 21 15:50:21.129548 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:50:21.129548 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:50:21.129548 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:50:21.129548 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:21.129841 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:50:21.130453 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.023(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:50:21.130792 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:50:22.132173 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:50:22.134006 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.145929 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.146224 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[502d5e41-eb7a-4df7-a0bf-440dcd2ce6fc]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:50:22.147219 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.150852 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.155290 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.007s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.155409 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a7eabd9-9583-420e-9d6a-d2143e1efd77]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.155860 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.156176 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.162886 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.006s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.163873 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a1307af-be30-4565-8376-3adb37083e88]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.164083 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.164808 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.auth.username -v oiEx9AfBKxyfq8m4B4wJ {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.174515 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.auth.username -v oiEx9AfBKxyfq8m4B4wJ" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.174688 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[aff0b537-8d78-41bf-8478-a43f3c9cd792]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.175106 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oiEx9AfBKxyfq8m4B4wJ'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.176131 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.185798 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.186380 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdc0563f-d5bf-4675-a956-43d86d287f5f]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.186862 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.187549 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.196515 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.196738 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1f667da-d638-4ab8-9f49-acb33e430741]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.197043 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:22.197043 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:22.197217 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:22.197217 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:50:22.197877 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.212618 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --login" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.212809 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdbe1fef-2d29-4714-8647-4fa45a604c6f]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.213128 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc, portal: 38.102.83.94,3260] successful. Nov 21 15:50:22.213128 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.214008 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.221542 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.007s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.221542 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ce456ca-c244-4c5a-ae18-b5a53a04b44a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.221542 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:22.222022 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.230394 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.230519 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[380ded89-d091-484b-8cd7-7290c9692666]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\ntcp: [29] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.230754 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:22.230754 np0005531460.novalocal cinder-backup[107086]: tcp: [29] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc (non-flash) Nov 21 15:50:22.230754 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:22.230963 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:22.230963 np0005531460.novalocal cinder-backup[107086]: tcp: [29] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc (non-flash) Nov 21 15:50:22.230963 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:50:22.231513 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.124s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:50:22.231696 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:50:22.232252 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] HCTL ('3', '-', '-', 0) found on session 29 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:50:22.232421 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:50:22.233007 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:22.251593 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.017s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:22.251593 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b97cd780-3d7e-4f67-a9b5-a2ced2aee680]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:22.252836 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 29 and hctl ['3', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:50:23.254804 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 29 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:50:23.255937 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'target_lun': 0, 'volume_id': 'ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'auth_method': 'CHAP', 'auth_username': 'oiEx9AfBKxyfq8m4B4wJ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:50:23.257118 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.163s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:50:23.258161 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2164ms) {'type': 'block', 'scsi_wwn': '360014059f616bcb1ae14da387d5bd36a', 'path': '/dev/sdb'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:50:23.260223 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:23.290330 np0005531460.novalocal sudo[148370]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 21 15:50:23.639020 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.376s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:23.641842 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:50:23.654041 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1902449308,backup: f2011b7c-6706-4260-a22e-269dff7e0233. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:50:23.734135 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] generate_object_name_prefix: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233 {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:50:23.739649 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] starting backup of volume: 6ef3b23a-f7b5-420d-86b8-2fb15f829882, volume size: 1073741824, object names prefix volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:50:23.987932 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:24.117439 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:24.117439 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:24.307652 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:24.308145 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:24.545537 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:24.712080 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:24.712374 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:24.864403 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:24.864665 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:25.041696 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:25.167843 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:25.168118 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:25.289495 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:25.289777 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:25.472521 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:25.606221 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:25.606357 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:25.747035 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:25.747146 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:26.358071 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:26.521140 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:26.521515 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:26.674957 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:26.674957 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:26.823188 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:26.950092 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:26.950358 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:27.123162 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:27.123431 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:27.339291 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:27.463095 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:27.463483 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:27.597079 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:27.597642 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:28.018981 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:28.155661 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:28.156055 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:28.325134 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:28.325405 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:28.472215 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:28.599547 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:28.600023 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:28.714080 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:28.714376 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:29.112530 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:29.276721 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:29.277020 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:29.418989 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:29.419123 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:29.561611 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:29.686913 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:29.687373 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:29.834873 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:29.835021 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:29.990232 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:30.124874 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:30.125017 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:30.285306 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:30.285587 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:30.454373 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:30.586254 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:30.586509 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:30.732965 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:30.732965 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:30.902733 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:31.018930 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:31.018930 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:31.189735 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:31.190040 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:31.407358 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:31.535824 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:31.536038 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:31.714724 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:31.714876 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:31.857426 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:31.988553 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:31.988676 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:32.134563 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:32.134663 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:32.344352 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:32.478701 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:32.479058 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:32.621204 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:32.621471 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:32.778459 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:32.914096 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:32.914490 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:33.063407 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:33.063586 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:33.643323 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:33.776256 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:33.776419 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:33.887798 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:33.888105 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:34.058601 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:34.191834 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:34.192102 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:34.371542 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:34.371888 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:34.472289 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:50:34.533360 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:50:34.533669 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:50:34.642996 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup MD5 for volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:50:34.643276 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:50:34.649279 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:50:34.661905 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:50:34.662355 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:50:34.668937 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:50:34.669289 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:50:34.675816 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:50:34.675816 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1902449308, sha256file filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:50:34.767495 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:50:34.768178 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1902449308, metadata filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:50:34.801956 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "f2011b7c-6706-4260-a22e-269dff7e0233",\n "backup_name": "tempest-VolumesBackupsTest-Backup-81611907",\n "created_at": "2025-11-21 20:50:16+00:00",\n "objects": [\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "6ef3b23a-f7b5-420d-86b8-2fb15f829882",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"6ef3b23a-f7b5-420d-86b8-2fb15f829882\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"78aab0b8a676406dab9c030d6a4b429f\\", \\"project_id\\": \\"99dea2ee54874291a7a3284a115f81e4\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:49:19.000000\\", \\"launched_at\\": \\"2025-11-21T20:49:20.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1746786993\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 0\\", \\"provider_auth\\": \\"CHAP cvJm23aL9yB2WqgmHZJ8 7fqJFTEN5ZGFMPYR\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:49:19.000000\\", \\"updated_at\\": \\"2025-11-21T20:50:16.000000\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:50:34.810628 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] backup f2011b7c-6706-4260-a22e-269dff7e0233 finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:50:34.901349 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:34.930261 np0005531460.novalocal sudo[148394]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 21 15:50:35.246014 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.344s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.247753 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'target_lun': 0, 'volume_id': 'ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 'auth_method': 'CHAP', 'auth_username': 'oiEx9AfBKxyfq8m4B4wJ', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014059f616bcb1ae14da387d5bd36a', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:50:35.248149 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:50:35.248872 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:50:35.251223 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.263181 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.263385 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7499309-a80a-4a64-b74f-6c4010346ecd]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:50:35.263941 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:35.264343 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:50:35.264343 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:50:35.264343 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:50:35.264716 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:50:35.265872 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.277380 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.277617 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8710ff0f-d49b-42e9-b000-124f46193af1]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882\n38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.278878 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.289651 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.290675 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[59b763d0-a306-4c0f-b67f-e772180b0a6f]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\ntcp: [29] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.291481 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:35.291481 np0005531460.novalocal cinder-backup[107086]: tcp: [29] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc (non-flash) Nov 21 15:50:35.291481 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:35.291958 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:35.291958 np0005531460.novalocal cinder-backup[107086]: tcp: [29] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc (non-flash) Nov 21 15:50:35.291958 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:50:35.293469 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb51f2520>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc'): ({'sdb'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:50:35.294511 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sdb {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:50:35.295573 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.302715 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[373ec0a3-27a8-48ec-86be-d6242f0ed018]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:50:35.304795 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:50:35.306592 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[373ec0a3-27a8-48ec-86be-d6242f0ed018]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.307537 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sdb {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:50:35.308505 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.316908 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.317316 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b8112cc-c2f6-426f-917b-9151192eb3d1]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.317737 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:50:35.319518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.363743 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.045s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.364314 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b04127cd-c6bb-4abe-9ca6-baca01c034b1]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.364893 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:50:35.365194 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] SCSI volumes sdb have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:50:35.365860 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:50:35.367221 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.379654 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.379938 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[71aba6fd-ebb2-4ac3-a91e-e87bf7aa50a4]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.380363 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:35.381487 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.413418 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --logout" returned: 0 in 0.031s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.413694 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ac4fda2-1d8f-4154-9a5d-31f0244418d7]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc, portal: 38.102.83.94,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.414352 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc, portal: 38.102.83.94,3260] Nov 21 15:50:35.414352 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc, portal: 38.102.83.94,3260] successful. Nov 21 15:50:35.414352 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:35.415377 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:35.428089 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea8bef0e-09ce-4148-bf03-cc1989bc00fc -p 38.102.83.94:3260 --op delete" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:35.428341 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[285593d0-ef96-49a2-9727-d9a45305db07]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:35.428743 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:35.429490 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.181s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:50:35.429809 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (181ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:50:38.982220 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-1db2df15-5778-4278-ab04-1135668120ac tempest-VolumesBackupsTest-633227990 None] Create backup finished. backup: f2011b7c-6706-4260-a22e-269dff7e0233. Nov 21 15:50:40.003336 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Create backup started, backup: f19e1e3b-6b83-4e54-8c59-9fc208aad6d1 volume: 6ef3b23a-f7b5-420d-86b8-2fb15f829882. Nov 21 15:50:40.007565 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container=None,created_at=2025-11-21T20:50:39Z,data_timestamp=2025-11-21T20:50:39Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=f19e1e3b-6b83-4e54-8c59-9fc208aad6d1,metadata={},num_dependent_backups=0,object_count=0,parent=Backup(f2011b7c-6706-4260-a22e-269dff7e0233),parent_id='f2011b7c-6706-4260-a22e-269dff7e0233',project_id='99dea2ee54874291a7a3284a115f81e4',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='78aab0b8a676406dab9c030d6a4b429f',volume_id=6ef3b23a-f7b5-420d-86b8-2fb15f829882) Nov 21 15:50:41.449691 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:50:41.449691 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:50:41.450230 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:50:41.452046 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:41.460558 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:41.460843 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b16c5b9-44e7-4204-a8aa-396261270a8c]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[b516caa8-364b-447c-a327-332609323770]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:41.463903 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:50:41.464198 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b516caa8-364b-447c-a327-332609323770]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:41.465419 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:50:41.465916 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:50:41.466751 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:41.476469 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:41.476704 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9329e78-5f18-4d4a-a19e-d8dda8ddae1a]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:41.477738 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:41.487810 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:41.488066 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[59796958-ff2f-4ec8-9dcb-f0e2da4871d4]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:41.489127 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd40ae38-98bd-46fa-8776-2b71cd99feb4]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:41.489617 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:41.506540 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:50:41.507017 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:50:41.509102 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:50:41.510223 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef8:dc6', 'fe80::34:b9ff:fe8e:4a20'] Nov 21 15:50:41.510474 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:50:41.510792 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:50:41.511358 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (60ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef8:dc6', 'fe80::34:b9ff:fe8e:4a20']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:50:45.019967 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:50:45.020630 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f', 'target_lun': 0, 'volume_id': '41fdbc52-9406-4c32-b506-22406d22141f', 'auth_method': 'CHAP', 'auth_username': 'N5wFG22ARsLqGdfhJvAG', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:50:45.020630 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:50:45.021019 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.000s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:50:45.021193 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f', 'target_lun': 0, 'volume_id': '41fdbc52-9406-4c32-b506-22406d22141f', 'auth_method': 'CHAP', 'auth_username': 'N5wFG22ARsLqGdfhJvAG', 'auth_password': '5C7cJAG6hqkPWuNY', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:50:45.022286 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:45.029820 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2bb258e5-6d3f-4487-8e94-1ebcbdd65d48]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:50:45.032015 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:50:45.034598 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bb258e5-6d3f-4487-8e94-1ebcbdd65d48]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:45.035153 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:50:45.035555 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.000s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:50:45.036030 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:50:45.036807 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:45.048055 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:45.048354 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b665fa7-de21-43bb-b921-61d3bf18a35a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:45.048608 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:50:45.048608 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:45.050037 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:45.061907 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:45.062321 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[79b27fd7-c729-475f-b794-509955844cae]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:45.062874 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f] added Nov 21 15:50:45.062874 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:50:45.062874 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:50:45.062874 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:50:45.062874 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:45.063139 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:50:45.063857 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.028(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:50:45.064204 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:50:46.065976 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:50:46.068328 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.080714 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.081176 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f777a25-ac79-4cdb-91ed-61e8411536b8]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:50:46.083208 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.085944 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.098624 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.098895 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[95dcfa4d-e4ec-4e3d-84b5-09349b1efd6c]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.099383 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.100393 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.111799 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.112057 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f08ac460-dcbe-41b7-b7ad-a53142ddbea3]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.112470 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.113488 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.auth.username -v N5wFG22ARsLqGdfhJvAG {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.124574 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.auth.username -v N5wFG22ARsLqGdfhJvAG" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.124813 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[34dfe895-70cd-4ebb-91e7-7998681869d3]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.125239 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'N5wFG22ARsLqGdfhJvAG'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.126396 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.137557 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.137734 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[859f2994-40ea-4891-84ad-7d9933eba342]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.138365 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.139427 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.152714 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.153101 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b01c56a-a94c-47b6-a037-bd001edf160d]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.153490 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:46.153490 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:46.153748 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:46.153748 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:50:46.154864 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.174040 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --login" returned: 0 in 0.019s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.174211 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c3389dc-1140-4ab5-98c7-97833fde560c]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.174657 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f, portal: 38.102.83.94,3260] successful. Nov 21 15:50:46.174657 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.175736 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.186836 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.187085 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b17ce434-ff3e-417b-b07f-cc98839a916a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.187517 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:46.188822 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.198209 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.198431 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ee4dadc-239a-4940-bb4c-18e70629ae3d]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\ntcp: [30] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.198845 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:46.198845 np0005531460.novalocal cinder-backup[107086]: tcp: [30] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f (non-flash) Nov 21 15:50:46.198845 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:46.199136 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:46.199136 np0005531460.novalocal cinder-backup[107086]: tcp: [30] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f (non-flash) Nov 21 15:50:46.199136 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:50:46.199691 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.164s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:50:46.199991 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:50:46.200797 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] HCTL ('3', '-', '-', 0) found on session 30 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:50:46.201095 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:50:46.202662 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:46.226672 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.024s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:46.226986 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[33b9a3a8-9f05-43ba-ab00-b0e39f308912]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:46.227760 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 30 and hctl ['3', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:50:46.494089 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:50:46.494397 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:50:47.230119 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 30 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:50:47.231808 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f', 'target_lun': 0, 'volume_id': '41fdbc52-9406-4c32-b506-22406d22141f', 'auth_method': 'CHAP', 'auth_username': 'N5wFG22ARsLqGdfhJvAG', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:50:47.231932 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.211s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:50:47.232276 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2211ms) {'type': 'block', 'scsi_wwn': '36001405504a6c3f5a9f462bb45f430b0', 'path': '/dev/sdb'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:50:47.233808 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:47.264661 np0005531460.novalocal sudo[148630]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 21 15:50:47.565946 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.332s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:47.576351 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:50:47.591102 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _read_sha256file started, container name: tempest-volumesbackupstest-backup-container-1902449308, sha256 filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_sha256file. {{(pid=107086) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:336}} Nov 21 15:50:47.634631 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _read_sha256file finished. {{(pid=107086) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:343}} Nov 21 15:50:47.649546 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _create_container started, container: volumebackups,backup: f19e1e3b-6b83-4e54-8c59-9fc208aad6d1. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:50:47.773452 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] generate_object_name_prefix: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1 {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:50:47.778901 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] starting backup of volume: 6ef3b23a-f7b5-420d-86b8-2fb15f829882, volume size: 1073741824, object names prefix volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:50:53.438414 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:50:53.451954 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:50:53.452119 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:50:53.458503 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:50:53.458694 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:50:53.463197 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:50:53.463664 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _write_sha256file started, container name: volumebackups, sha256file filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:50:53.563922 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:50:53.564845 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _write_metadata started, container name: volumebackups, metadata filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:50:53.598901 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "f19e1e3b-6b83-4e54-8c59-9fc208aad6d1",\n "backup_name": null,\n "created_at": "2025-11-21 20:50:39+00:00",\n "objects": [],\n "parent_id": "f2011b7c-6706-4260-a22e-269dff7e0233",\n "version": "1.0.0",\n "volume_id": "6ef3b23a-f7b5-420d-86b8-2fb15f829882",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"6ef3b23a-f7b5-420d-86b8-2fb15f829882\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"78aab0b8a676406dab9c030d6a4b429f\\", \\"project_id\\": \\"99dea2ee54874291a7a3284a115f81e4\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:49:19.000000\\", \\"launched_at\\": \\"2025-11-21T20:49:20.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1746786993\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 0\\", \\"provider_auth\\": \\"CHAP cvJm23aL9yB2WqgmHZJ8 7fqJFTEN5ZGFMPYR\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:49:19.000000\\", \\"updated_at\\": \\"2025-11-21T20:50:39.000000\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:50:53.603709 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] backup f19e1e3b-6b83-4e54-8c59-9fc208aad6d1 finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:50:53.699304 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:53.732311 np0005531460.novalocal sudo[148654]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 21 15:50:54.059574 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.360s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.061255 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f', 'target_lun': 0, 'volume_id': '41fdbc52-9406-4c32-b506-22406d22141f', 'auth_method': 'CHAP', 'auth_username': 'N5wFG22ARsLqGdfhJvAG', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405504a6c3f5a9f462bb45f430b0', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:50:54.061590 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:50:54.062318 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:50:54.063950 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.076619 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.076938 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5500bf79-c6fb-4aa1-b17d-c854b9518baa]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:50:54.077399 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:54.077830 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:50:54.077830 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:50:54.077830 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:50:54.078234 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:50:54.079197 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.090371 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.090636 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d21e8560-b1d0-47bd-a289-7ca523248bde]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f\n38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.091638 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.105668 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.014s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.106011 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ca003d7-ea2c-45a4-97bc-9571d5bf9546]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\ntcp: [30] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.106484 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:54.106484 np0005531460.novalocal cinder-backup[107086]: tcp: [30] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f (non-flash) Nov 21 15:50:54.106484 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:50:54.106735 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:50:54.106735 np0005531460.novalocal cinder-backup[107086]: tcp: [30] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f (non-flash) Nov 21 15:50:54.106735 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:50:54.107679 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb84ddee0>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f'): ({'sdb'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:50:54.108092 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sdb {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:50:54.110414 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.116475 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[8a676331-5ab6-410a-bc41-6a0d7b3daf5c]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:50:54.119800 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:50:54.123043 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a676331-5ab6-410a-bc41-6a0d7b3daf5c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.123043 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sdb {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:50:54.124168 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.135080 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.135458 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bcc530d-3d86-46f9-8663-95db99866f8a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.135974 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:50:54.136981 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.197900 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.060s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.200604 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8ad0230-2eb8-4102-b8c0-85728a25889e]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.200604 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:50:54.200604 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] SCSI volumes sdb have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:50:54.200604 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:50:54.201844 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.218414 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.016s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.218414 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[572a826f-36b5-4389-bf84-37786c104d95]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.220222 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:54.220222 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.249697 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --logout" returned: 0 in 0.029s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.250007 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[93625384-3954-4548-882e-60e57056a1bd]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f, portal: 38.102.83.94,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.250516 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f, portal: 38.102.83.94,3260] Nov 21 15:50:54.250516 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f, portal: 38.102.83.94,3260] successful. Nov 21 15:50:54.250516 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:54.251590 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:50:54.266648 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-41fdbc52-9406-4c32-b506-22406d22141f -p 38.102.83.94:3260 --op delete" returned: 0 in 0.015s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:50:54.266926 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[27587ea0-12bf-43b0-a831-b9c451d6bf49]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:50:54.267326 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:50:54.267791 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.205s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:50:54.268092 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (206ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:50:57.824940 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-f4e184f8-f3c9-4528-be8d-5317ec397660 tempest-VolumesBackupsTest-633227990 None] Create backup finished. backup: f19e1e3b-6b83-4e54-8c59-9fc208aad6d1. Nov 21 15:50:58.733480 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] Delete backup started, backup: f19e1e3b-6b83-4e54-8c59-9fc208aad6d1. Nov 21 15:50:58.737922 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:50:58.738224 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:50:58.738524 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] delete started, backup: f19e1e3b-6b83-4e54-8c59-9fc208aad6d1, container: volumebackups, prefix: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:50:58.799452 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1_metadata', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:50:58.821225 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1_metadata in container: volumebackups. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:50:58.845097 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205047/az_nova_backup_f19e1e3b-6b83-4e54-8c59-9fc208aad6d1_sha256file in container: volumebackups. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:50:58.845345 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] delete f19e1e3b-6b83-4e54-8c59-9fc208aad6d1 finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:50:58.848823 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:50:58.874495 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] Created reservations ['2191d9fd-e9b6-476a-8f4c-3c4a139f2ac0', '0b9b850e-2d6d-40e0-8bdc-23d9596c050b'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:50:58.907407 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-c494899f-bc96-4743-8327-ca00c3afb0ec tempest-VolumesBackupsTest-633227990 None] Delete backup finished, backup f19e1e3b-6b83-4e54-8c59-9fc208aad6d1 deleted. Nov 21 15:50:59.944046 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Create backup started, backup: 35174d98-ae16-4ba1-9679-77b613da6182 volume: 6ef3b23a-f7b5-420d-86b8-2fb15f829882. Nov 21 15:50:59.950889 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-442682006',created_at=2025-11-21T20:50:59Z,data_timestamp=2025-11-21T20:50:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1697498241',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=35174d98-ae16-4ba1-9679-77b613da6182,metadata={},num_dependent_backups=0,object_count=0,parent=Backup(f2011b7c-6706-4260-a22e-269dff7e0233),parent_id='f2011b7c-6706-4260-a22e-269dff7e0233',project_id='99dea2ee54874291a7a3284a115f81e4',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='78aab0b8a676406dab9c030d6a4b429f',volume_id=6ef3b23a-f7b5-420d-86b8-2fb15f829882) Nov 21 15:51:01.392920 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:51:01.393368 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:51:01.393807 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:51:01.395118 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:01.404481 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:01.404953 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[edd56d22-7739-4072-a058-cc686383d3b5]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[cb54f839-6d71-4622-ac22-0c974d222df4]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:01.407497 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:51:01.408373 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb54f839-6d71-4622-ac22-0c974d222df4]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:01.409032 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:51:01.409508 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:51:01.410403 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:01.420223 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:01.420483 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1031e64-1141-4c9d-b823-9184ddd64a4a]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:01.421500 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:01.431954 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:01.432406 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[08f96093-120f-445c-83b9-e5a19b9d221b]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:01.433508 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1df733f-28fc-4455-ab60-ee386eef3b0c]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:01.434041 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:01.450200 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:51:01.450600 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:51:01.452650 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:51:01.453708 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef8:dc6', 'fe80::34:b9ff:fe8e:4a20'] Nov 21 15:51:01.454016 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:51:01.454272 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:51:01.454837 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] <== get_connector_properties: return (60ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944', 'fe80::fc16:3eff:fef8:dc6', 'fe80::34:b9ff:fe8e:4a20']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:51:04.964889 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:51:04.966157 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'target_lun': 0, 'volume_id': '1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'auth_method': 'CHAP', 'auth_username': 'E3tjbrDbw2tVBzHeGV8W', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:51:04.966467 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:51:04.967153 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:51:04.967438 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'target_lun': 0, 'volume_id': '1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'auth_method': 'CHAP', 'auth_username': 'E3tjbrDbw2tVBzHeGV8W', 'auth_password': '8DBGF6HPWF5q5xPB', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:51:04.968656 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:04.977370 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[1b0e1195-c1ac-4523-90cb-cd39ef8beb77]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:51:04.979507 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:51:04.981324 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b0e1195-c1ac-4523-90cb-cd39ef8beb77]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:04.982114 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:51:04.982716 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:51:04.983107 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:51:04.985153 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:04.996219 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260" returned: 21 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:04.997112 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbe9ed31-1043-44ab-9df0-11ecafaf275a]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:04.997517 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:51:04.997517 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:04.998803 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:05.010661 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:05.011152 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[52616999-a0d8-4be2-996a-38601407a2ae]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:05.011842 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854] added Nov 21 15:51:05.011842 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:51:05.011842 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:51:05.011842 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:51:05.011842 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:05.012349 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:51:05.013254 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.030(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:51:05.013688 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:51:06.015100 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:51:06.016524 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.027490 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.028245 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[e96769e0-14e1-4805-ad9d-4d3d4525de38]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:51:06.029988 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.032661 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.043140 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.043468 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf6f4562-5deb-46e7-8454-0a88b0f9d978]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.043897 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.044787 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.056374 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.056626 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f96db81-3ee4-4db7-b501-630d33ee7ffb]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.057057 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.057963 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v E3tjbrDbw2tVBzHeGV8W {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.071373 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v E3tjbrDbw2tVBzHeGV8W" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.071630 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d570eb05-7db7-47c8-8066-80b8ed5b33ff]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.072066 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'E3tjbrDbw2tVBzHeGV8W'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.073372 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.085967 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.086197 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfd24c6f-dbd8-429a-8b5c-29fc2c068b59]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.086727 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.087856 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.100006 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.100327 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6075f140-21c4-4f23-9ad8-efcc41c7de08]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.100829 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:51:06.100829 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:06.101222 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:51:06.101222 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:51:06.102850 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.125545 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --login" returned: 0 in 0.023s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.125868 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0729a09f-949a-494b-b4d3-06d4a1935d8d]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.126463 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854, portal: 38.102.83.94,3260] successful. Nov 21 15:51:06.126463 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.127518 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.139265 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.139542 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b15e6c86-e24d-4e62-95f3-cbbd1136ef62]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.139971 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:06.140872 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.154318 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.154606 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7461eac-aa16-4589-a5bb-609419e1169a]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\ntcp: [31] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.155069 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:51:06.155069 np0005531460.novalocal cinder-backup[107086]: tcp: [31] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 (non-flash) Nov 21 15:51:06.155069 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:06.155329 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:51:06.155329 np0005531460.novalocal cinder-backup[107086]: tcp: [31] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 (non-flash) Nov 21 15:51:06.155329 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:51:06.155889 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.173s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:51:06.156160 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:51:06.156974 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] HCTL ('3', '-', '-', 0) found on session 31 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:51:06.157240 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:51:06.158256 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:06.184140 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.026s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:06.184463 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ea14297-85e1-427f-803e-55fc579766a3]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:06.185113 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 31 and hctl ['3', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:51:07.187314 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Searching for a device in session 31 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:51:07.188155 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'target_lun': 0, 'volume_id': '1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'auth_method': 'CHAP', 'auth_username': 'E3tjbrDbw2tVBzHeGV8W', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:51:07.189116 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.222s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:51:07.189447 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] <== connect_volume: return (2222ms) {'type': 'block', 'scsi_wwn': '36001405c024014e39e94f20ba7647358', 'path': '/dev/sdb'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:51:07.190088 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:07.218009 np0005531460.novalocal sudo[148906]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 21 15:51:07.557522 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.367s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:07.561988 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:51:07.563134 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=84,cinder:UPDATE=23,cinder:INSERT=2 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:51:07.567397 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _read_sha256file started, container name: tempest-volumesbackupstest-backup-container-1902449308, sha256 filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_sha256file. {{(pid=107086) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:336}} Nov 21 15:51:07.607211 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _read_sha256file finished. {{(pid=107086) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:343}} Nov 21 15:51:07.618288 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _create_container started, container: tempest-volumesbackupstest-backup-container-442682006,backup: 35174d98-ae16-4ba1-9679-77b613da6182. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:51:07.680812 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] generate_object_name_prefix: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182 {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:51:07.684857 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] starting backup of volume: 6ef3b23a-f7b5-420d-86b8-2fb15f829882, volume size: 1073741824, object names prefix volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:51:13.702840 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:51:13.715161 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:51:13.715475 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:51:13.720818 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:51:13.721120 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:51:13.726168 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:51:13.726658 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-442682006, sha256file filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:51:13.819680 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:51:13.820367 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-442682006, metadata filename: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:51:13.858822 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "35174d98-ae16-4ba1-9679-77b613da6182",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1697498241",\n "created_at": "2025-11-21 20:50:59+00:00",\n "objects": [],\n "parent_id": "f2011b7c-6706-4260-a22e-269dff7e0233",\n "version": "1.0.0",\n "volume_id": "6ef3b23a-f7b5-420d-86b8-2fb15f829882",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"6ef3b23a-f7b5-420d-86b8-2fb15f829882\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"78aab0b8a676406dab9c030d6a4b429f\\", \\"project_id\\": \\"99dea2ee54874291a7a3284a115f81e4\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:49:19.000000\\", \\"launched_at\\": \\"2025-11-21T20:49:20.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1746786993\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 0\\", \\"provider_auth\\": \\"CHAP cvJm23aL9yB2WqgmHZJ8 7fqJFTEN5ZGFMPYR\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:49:19.000000\\", \\"updated_at\\": \\"2025-11-21T20:50:59.000000\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:51:13.865120 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] backup 35174d98-ae16-4ba1-9679-77b613da6182 finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:51:13.956794 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:13.987404 np0005531460.novalocal sudo[148922]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 21 15:51:14.299459 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.342s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.301124 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'target_lun': 0, 'volume_id': '1415a98f-3a54-47f1-b4c6-49cb78a2a854', 'auth_method': 'CHAP', 'auth_username': 'E3tjbrDbw2tVBzHeGV8W', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405c024014e39e94f20ba7647358', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:51:14.301443 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:51:14.302131 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:51:14.303932 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.312599 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fb1c4af-cca0-4042-adeb-69791b3bb67a]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:51:14.313930 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:51:14.314484 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:51:14.315535 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.322611 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.007s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.322807 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0337c21-0f01-4243-b283-114cb38d370f]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854\n38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.323857 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.331925 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.332245 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[601c8b47-2f30-4eb4-a34c-bdd69a157db4]: (4, ('tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash)\ntcp: [31] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.332658 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:51:14.332658 np0005531460.novalocal cinder-backup[107086]: tcp: [31] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 (non-flash) Nov 21 15:51:14.332658 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:14.332942 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsi session list stdout=tcp: [28] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-6ef3b23a-f7b5-420d-86b8-2fb15f829882 (non-flash) Nov 21 15:51:14.332942 np0005531460.novalocal cinder-backup[107086]: tcp: [31] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 (non-flash) Nov 21 15:51:14.332942 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:51:14.333906 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Resulting device map defaultdict(. at 0x7fddb5213b00>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854'): ({'sdb'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:51:14.334315 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Removing single pathed devices sdb {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:51:14.335202 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.340261 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[bb57b354-5f03-4014-b314-f11457ef8576]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:51:14.342462 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:51:14.344155 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb57b354-5f03-4014-b314-f11457ef8576]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.344757 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Flushing IO for device /dev/sdb {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:51:14.345638 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.353460 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.353824 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4b54244-a5d5-4a03-a2ef-a397b49be3ac]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.354128 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:51:14.355038 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.393683 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.038s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.394427 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a332e247-b6b2-43a2-aa11-c4cf20480ae3]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.395078 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:51:14.395398 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] SCSI volumes sdb have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:51:14.396032 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:51:14.397428 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.410397 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.410581 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a11c29fb-cc48-4214-b838-26f6cfd65751]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.411130 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:14.412194 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.447434 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --logout" returned: 0 in 0.035s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.447798 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[525f2125-9641-4d48-b09c-7602a4b8b338]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854, portal: 38.102.83.94,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.448293 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854, portal: 38.102.83.94,3260] Nov 21 15:51:14.448293 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854, portal: 38.102.83.94,3260] successful. Nov 21 15:51:14.448293 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:14.449319 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:14.461565 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-1415a98f-3a54-47f1-b4c6-49cb78a2a854 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:14.461843 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[d96515cb-a7ff-40d5-be55-57d9e1e37e92]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:14.462311 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:14.462932 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.161s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:51:14.463360 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] <== disconnect_volume: return (161ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:51:18.032505 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-ae428c55-2623-4be7-96e0-a92e6273eed5 tempest-VolumesBackupsTest-633227990 None] Create backup finished. backup: 35174d98-ae16-4ba1-9679-77b613da6182. Nov 21 15:51:18.677042 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] Delete backup started, backup: 35174d98-ae16-4ba1-9679-77b613da6182. Nov 21 15:51:18.678124 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:51:18.678292 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:51:18.678476 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] delete started, backup: 35174d98-ae16-4ba1-9679-77b613da6182, container: tempest-volumesbackupstest-backup-container-442682006, prefix: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:51:18.743654 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182_metadata', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:51:18.774233 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182_metadata in container: tempest-volumesbackupstest-backup-container-442682006. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:18.804805 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205107/az_nova_backup_35174d98-ae16-4ba1-9679-77b613da6182_sha256file in container: tempest-volumesbackupstest-backup-container-442682006. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:18.805319 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] delete 35174d98-ae16-4ba1-9679-77b613da6182 finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:51:18.811747 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:51:18.845419 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] Created reservations ['0f46abf4-bc62-4dfc-a9c8-f797c6449468', '9a3e9f2c-354c-4ae7-9565-7ec2dffa5cb2'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:51:18.875987 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-8a97a4b4-1ac1-403e-af29-8f1080df0369 tempest-VolumesBackupsTest-633227990 None] Delete backup finished, backup 35174d98-ae16-4ba1-9679-77b613da6182 deleted. Nov 21 15:51:19.879244 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] Delete backup started, backup: f2011b7c-6706-4260-a22e-269dff7e0233. Nov 21 15:51:19.881821 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_99dea2ee54874291a7a3284a115f81e4 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:51:19.881821 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:51:19.881821 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] delete started, backup: f2011b7c-6706-4260-a22e-269dff7e0233, container: tempest-volumesbackupstest-backup-container-1902449308, prefix: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:51:19.942489 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] generated object list: ['volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00001', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00002', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00003', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00004', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00005', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00006', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00007', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00008', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00009', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00010', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00011', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00012', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00013', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00014', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00015', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00016', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00017', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00018', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00019', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00020', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00021', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_metadata', 'volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:51:19.975230 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00001 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.009991 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00002 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.037384 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00003 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.060862 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00004 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.085992 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00005 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.112449 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00006 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.141067 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00007 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.167165 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00008 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.192892 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00009 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.219421 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00010 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.249042 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00011 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.275181 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00012 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.298043 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00013 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.323002 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00014 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.352036 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00015 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.380179 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00016 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.407899 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00017 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.435392 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00018 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.461483 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00019 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.493695 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00020 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.518851 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233-00021 in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.547245 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_metadata in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.573631 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] deleted object: volume_6ef3b23a-f7b5-420d-86b8-2fb15f829882/20251121205023/az_nova_backup_f2011b7c-6706-4260-a22e-269dff7e0233_sha256file in container: tempest-volumesbackupstest-backup-container-1902449308. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:51:20.574032 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] delete f2011b7c-6706-4260-a22e-269dff7e0233 finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:51:20.577752 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:51:20.608425 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] Created reservations ['b6203773-fc56-4423-8c49-3e1e0c4da8ed', 'c1683b80-7ead-4158-992a-8da032669533'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:51:20.633161 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-a9968447-35d1-43e9-96d8-df54618c0a60 tempest-VolumesBackupsTest-633227990 None] Delete backup finished, backup f2011b7c-6706-4260-a22e-269dff7e0233 deleted. Nov 21 15:51:30.629797 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=46,cinder:UPDATE=19,cinder:INSERT=4 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:51:40.600310 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Create backup started, backup: 6351a90a-2902-461c-96c0-a2f99d513954 volume: 3cd73553-e400-4d34-9879-a5a0d634cd06. Nov 21 15:51:40.604435 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupsv39test-backup-container-2111697882',created_at=2025-11-21T20:51:40Z,data_timestamp=2025-11-21T20:51:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-58732884',encryption_key_id=None,fail_reason=None,host='np0005531460.novalocal',id=6351a90a-2902-461c-96c0-a2f99d513954,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='aded80cc887d4fdc84d1c8adc45b82e7',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='548656622e944a32a7992a7e8cd0b0bf',volume_id=3cd73553-e400-4d34-9879-a5a0d634cd06) Nov 21 15:51:40.640551 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_aded80cc887d4fdc84d1c8adc45b82e7 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:51:40.640864 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:51:40.641356 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.94', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:51:40.642615 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:40.650717 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:40.651061 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[196ef693-086c-4028-9201-67e0b6f9e5bf]: (4, ('InitiatorName=iqn.1994-05.com.redhat:1e2adedf8ac\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[36ca3632-9584-4c8c-bebc-01f4c66a5353]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: with open_scini_device() as fd: Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: return next(self.gen) Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^ Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:40.654010 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 21 15:51:40.654575 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[36ca3632-9584-4c8c-bebc-01f4c66a5353]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:40.654742 np0005531460.novalocal cinder-backup[107086]: ERROR os_brick.initiator.connectors.scaleio [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 21 15:51:40.655032 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.scaleio [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 21 15:51:40.655804 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:40.665266 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:40.665569 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c4e81b3-65ab-474f-9b37-b6244c87297a]: (4, ('/dev/vda2\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:40.666423 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:40.676944 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:40.677694 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc9532db-27a0-4809-a348-bbd3317741c0]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:40.678849 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[526e9965-2447-4532-b80a-1da5daa07983]: (4, '46a3412e-8df5-4405-a700-00fd08a14826') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:40.679417 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Running cmd (subprocess): nvme version {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:40.695109 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] 'nvme version' failed. Not Retrying. {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:51:40.695499 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.nvmeof [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] nvme not present on system {{(pid=107086) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 21 15:51:40.697616 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107086) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 21 15:51:40.698555 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.lightos [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944'] Nov 21 15:51:40.698843 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 21 15:51:40.699099 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.lightos [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107086) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 21 15:51:40.699619 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] <== get_connector_properties: return (57ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.94', 'host': 'np0005531460.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:1e2adedf8ac', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '46a3412e-8df5-4405-a700-00fd08a14826', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.94', 'fe80::f816:3eff:fe52:cbef', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::a068:9cff:fe5d:6944']} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:51:44.213962 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connector [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Factory for iscsi on None {{(pid=107086) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 21 15:51:44.215296 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06', 'target_lun': 0, 'volume_id': '3cd73553-e400-4d34-9879-a5a0d634cd06', 'auth_method': 'CHAP', 'auth_username': 'ZMsY8GmG9HLYwm6q7CW4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:51:44.215568 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:51:44.216287 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:51:44.216574 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06', 'target_lun': 0, 'volume_id': '3cd73553-e400-4d34-9879-a5a0d634cd06', 'auth_method': 'CHAP', 'auth_username': 'ZMsY8GmG9HLYwm6q7CW4', 'auth_password': '45VvHoVLyBwLXhvK', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107086) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 21 15:51:44.217809 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:44.225214 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[66e71c83-35bf-42b3-b609-4a27035392db]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:51:44.227345 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:51:44.229020 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[66e71c83-35bf-42b3-b609-4a27035392db]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:44.229917 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:51:44.230596 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:51:44.230999 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:51:44.232049 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:44.242706 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260" returned: 21 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:44.242989 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d5fb004-ae87-4a94-931b-a8890d5f061f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:44.243447 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 21 15:51:44.243447 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:44.244506 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --interface default --op new {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:44.255655 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --interface default --op new" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:44.255963 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcfc116f-9c6f-4d3a-8879-1e38ceaea9ef]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:44.256406 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.94,3260,-1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06] added Nov 21 15:51:44.256406 np0005531460.novalocal cinder-backup[107086]: stderr=iscsiadm: Config file line 381 does not have value Nov 21 15:51:44.256406 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:51:44.256406 np0005531460.novalocal cinder-backup[107086]: iscsiadm: Config file line 381 does not have value Nov 21 15:51:44.256406 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:44.256672 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Retrying to connect to iSCSI portal 38.102.83.94:3260 {{(pid=107086) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 21 15:51:44.257327 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.026(s), this was the 1st time calling it. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 21 15:51:44.257650 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.utils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107086) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 21 15:51:45.259938 np0005531460.novalocal cinder-backup[107086]: INFO os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Trying to connect to iSCSI portal 38.102.83.94:3260 Nov 21 15:51:45.263522 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.273913 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.274875 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[cad8fa2b-8515-4d4e-894a-4773f2f0f7df]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.94\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.name = iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.tpgt = -1 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.startup = automatic Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.leading_login = No Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.iscsi_ifacename = default Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.net_ifacename = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.ipaddress = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.prefix_len = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.hwaddress = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.transport_name = tcp Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.initiatorname = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.vlan_id = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.vlan_priority = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.vlan_state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.iface_num = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.mtu = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.port = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.bootproto = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.subnet_mask = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.gateway = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id_state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_alt_client_id = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_dns = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_learn_iqn = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_req_vendor_id_state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id_state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_vendor_id = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.dhcp_slp_da = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.fragmentation = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.gratuitous_arp = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.incoming_forwarding = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tos_state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tos = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.ttl = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.delayed_ack = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tcp_nagle = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf_state = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tcp_wsf = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timer_scale = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.tcp_timestamp = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.redirect = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.def_task_mgmt_timeout = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.header_digest = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.data_digest = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.immediate_data = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.initial_r2t = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.data_seq_inorder = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.data_pdu_inorder = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.erl = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.max_receive_data_len = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.first_burst_len = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.max_outstanding_r2t = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.max_burst_len = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.chap_auth = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.bidi_chap = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.strict_login_compliance = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.discovery_auth = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: iface.discovery_logout = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.discovery_address = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.discovery_port = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.discovery_type = fw Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.initial_cmdsn = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.initial_login_retry_max = 8 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.xmit_thread_priority = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.cmds_max = 128 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.queue_depth = 32 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.nr_sessions = 1 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.auth.authmethod = None Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password = *** Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.auth.username_in = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.auth.password_in = Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.timeo.replacement_timeout = 120 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.abort_timeout = 15 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.lu_reset_timeout = 30 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.err_timeo.host_reset_timeout = 60 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FastAbort = Yes Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.InitialR2T = No Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ImmediateData = Yes Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.FirstBurstLength = 262144 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxBurstLength = 16776192 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Retain = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.DefaultTime2Wait = 2 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxConnections = 1 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.iscsi.ERL = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.scan = auto Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.reopen_max = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.session.conn_reopen_log_freq = 1 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].address = 38.102.83.94 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].port = 3260 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].startup = manual Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.window_size = 524288 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].tcp.type_of_service = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.logout_timeout = 15 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.login_timeout = 15 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.auth_timeout = 45 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_interval = 5 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].timeo.noop_out_timeout = 5 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.HeaderDigest = None Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.DataDigest = None Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.IFMarker = No Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: node.conn[0].iscsi.OFMarker = No Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: # END RECORD Nov 21 15:51:45.277069 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.279742 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.290288 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.290586 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[c61ac4d5-0b32-4092-97e4-7068bb80e905]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.291206 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.292323 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.305175 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.012s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.305499 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[b93eae48-c51c-4c72-bdc2-76c09d87455d]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.306109 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.307128 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v ZMsY8GmG9HLYwm6q7CW4 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.318220 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.auth.username -v ZMsY8GmG9HLYwm6q7CW4" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.318422 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[75987400-a4a1-4ea4-a5da-b31969fd8465]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.318900 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ZMsY8GmG9HLYwm6q7CW4'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.320052 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v *** {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.331573 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.331881 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[6abf4c1c-997e-43e3-ae4d-38c4afce1722]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.332442 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.333350 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.342285 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.009s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.342525 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bbaa8a3-8241-4c21-8afa-8179f262f15a]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.342948 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 21 15:51:45.342948 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:45.343199 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 21 15:51:45.343199 np0005531460.novalocal cinder-backup[107086]: {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:51:45.343432 np0005531460.novalocal cinder-backup[107086]: WARNING os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 21 15:51:45.343432 np0005531460.novalocal cinder-backup[107086]: Nov 21 15:51:45.344367 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --login {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.363752 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --login" returned: 0 in 0.018s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.364505 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f3dc007-12a5-4ad7-80bf-8629f91466f2]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.365220 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06, portal: 38.102.83.94,3260] successful. Nov 21 15:51:45.365220 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.366344 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.startup -v automatic {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.379416 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.startup -v automatic" returned: 0 in 0.013s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.379758 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[79ff67be-fc6a-4925-8238-32f144fb9c58]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.380312 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:45.381060 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.392411 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.392618 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[5351035d-3f4f-47c2-a5a3-0df6697396f8]: (4, ('tcp: [32] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.393054 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('-m', 'session'): stdout=tcp: [32] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 (non-flash) Nov 21 15:51:45.393054 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:45.393318 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsi session list stdout=tcp: [32] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 (non-flash) Nov 21 15:51:45.393318 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:51:45.394001 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Lock "connect_to_iscsi_portal-38.102.83.94:3260-iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.163s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:51:45.394279 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Connected to 38.102.83.94:3260 {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 21 15:51:45.395035 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] HCTL ('2', '-', '-', 0) found on session 32 with lun 0 {{(pid=107086) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 21 15:51:45.395308 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107086) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 21 15:51:45.396305 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:45.421107 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.025s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:45.421398 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cf504f6-367a-41a1-a01e-9ff6d752a8d5]: (4, ('- - 0', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:45.422090 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Searching for a device in session 32 and hctl ['2', '*', '*', 0] yield: None {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:51:46.424105 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Searching for a device in session 32 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107086) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 21 15:51:46.425093 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06', 'target_lun': 0, 'volume_id': '3cd73553-e400-4d34-9879-a5a0d634cd06', 'auth_method': 'CHAP', 'auth_username': 'ZMsY8GmG9HLYwm6q7CW4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107086) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 21 15:51:46.425915 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.210s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:51:46.426246 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] <== connect_volume: return (2210ms) {'type': 'block', 'scsi_wwn': '3600140535b44c47f5b64c4281b520a4b', 'path': '/dev/sda'} {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:51:46.426944 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:46.456842 np0005531460.novalocal sudo[149304]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 21 15:51:46.499943 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:51:46.500181 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:51:46.757619 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.330s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:46.764177 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:51:46.776824 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] _create_container started, container: tempest-volumesbackupsv39test-backup-container-2111697882,backup: 6351a90a-2902-461c-96c0-a2f99d513954. {{(pid=107086) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 21 15:51:47.027574 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] generate_object_name_prefix: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954 {{(pid=107086) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 21 15:51:47.035044 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] starting backup of volume: 3cd73553-e400-4d34-9879-a5a0d634cd06, volume size: 1073741824, object names prefix volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954, availability zone: nova {{(pid=107086) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 21 15:51:47.290516 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:47.419952 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:47.420064 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:47.584803 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:47.585261 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:47.745231 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:47.882874 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:47.882984 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:48.031556 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:48.031721 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:48.174238 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:48.316331 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:48.316436 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:48.421650 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:48.421970 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:48.596473 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:48.732099 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:48.732304 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:48.881492 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:48.881765 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:49.130114 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:49.265274 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:49.265368 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:49.409559 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:49.409863 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:49.580758 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:49.696510 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:49.697009 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:49.845459 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:49.845793 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:49.989858 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:50.166862 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:50.166862 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:50.319737 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:50.319737 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:50.504513 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:50.625104 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:50.625493 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:50.803346 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:50.803710 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:51.184792 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:51.352019 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:51.352304 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:51.509858 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:51.510133 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:51.725308 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:51.929513 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:51.929890 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:52.084933 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:52.085197 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:52.514422 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:52.650087 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:52.650348 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:52.805062 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:52.805318 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:52.965128 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:53.072937 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:53.073191 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:53.183013 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:53.183262 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:53.359583 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:53.504734 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:53.505043 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:53.657891 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:53.658147 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:54.016787 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:54.151867 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:54.152038 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:54.300477 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:54.300747 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:54.467476 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:54.613222 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:54.613487 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:54.755614 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:54.755614 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:54.922998 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:55.079365 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:55.079696 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:55.202415 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:55.202688 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:55.369682 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:55.473673 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:55.473964 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:55.621200 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:55.621307 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:55.775357 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:55.885646 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:55.885931 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:56.001070 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:56.001328 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:56.173883 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:56.286418 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:56.287606 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:56.447629 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:56.448058 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:56.830478 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:57.017587 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:57.017745 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:57.167587 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:57.167587 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:57.254587 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Backing up chunk of data from volume. {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 21 15:51:57.342125 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107086) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 21 15:51:57.342301 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] About to put_object {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 21 15:51:57.446918 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup MD5 for volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 21 15:51:57.447092 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Calling eventlet.sleep(0) {{(pid=107086) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 21 15:51:57.451451 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Getting metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 21 15:51:57.460071 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107086) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 21 15:51:57.460746 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Getting metadata type 'volume-metadata' {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 21 15:51:57.465696 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] No metadata type 'volume-metadata' available {{(pid=107086) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 21 15:51:57.465887 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Getting metadata type 'volume-glance-metadata' {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 21 15:51:57.468864 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.driver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] No metadata type 'volume-glance-metadata' available {{(pid=107086) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 21 15:51:57.469130 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] _write_sha256file started, container name: tempest-volumesbackupsv39test-backup-container-2111697882, sha256file filename: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954_sha256file. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 21 15:51:57.545993 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] _write_sha256file finished. {{(pid=107086) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 21 15:51:57.546555 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] _write_metadata started, container name: tempest-volumesbackupsv39test-backup-container-2111697882, metadata filename: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954_metadata. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 21 15:51:57.583207 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "6351a90a-2902-461c-96c0-a2f99d513954",\n "backup_name": "tempest-VolumesBackupsV39Test-Backup-58732884",\n "created_at": "2025-11-21 20:51:40+00:00",\n "objects": [\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "3cd73553-e400-4d34-9879-a5a0d634cd06",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"3cd73553-e400-4d34-9879-a5a0d634cd06\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"548656622e944a32a7992a7e8cd0b0bf\\", \\"project_id\\": \\"aded80cc887d4fdc84d1c8adc45b82e7\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005531460.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-21T20:51:39.000000\\", \\"launched_at\\": \\"2025-11-21T20:51:40.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsV39Test-Volume-1765579477\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.94:3260,iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 0\\", \\"provider_auth\\": \\"CHAP ZMsY8GmG9HLYwm6q7CW4 45VvHoVLyBwLXhvK\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"099441c3-a89d-49d9-8e0f-b86f1daa5de4\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"556adf35-c5a6-4979-a942-981586090f3a\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-21T20:51:39.000000\\", \\"updated_at\\": \\"2025-11-21T20:51:43.000000\\"}}"\n}'. {{(pid=107086) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 21 15:51:57.589517 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] backup 6351a90a-2902-461c-96c0-a2f99d513954 finished. {{(pid=107086) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 21 15:51:57.679402 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:57.709662 np0005531460.novalocal sudo[149343]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 21 15:51:58.020108 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.340s {{(pid=107086) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.021716 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.94:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06', 'target_lun': 0, 'volume_id': '3cd73553-e400-4d34-9879-a5a0d634cd06', 'auth_method': 'CHAP', 'auth_username': 'ZMsY8GmG9HLYwm6q7CW4', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '3600140535b44c47f5b64c4281b520a4b', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 21 15:51:58.022080 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 21 15:51:58.022745 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 21 15:51:58.024198 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.034257 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.034590 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[82c11aab-9cc3-4756-aeab-f33119df0a93]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06\n\tPortal: 38.102.83.94:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: iSNS: Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: STATIC: Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: Target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: Portal: 38.102.83.94:3260,-1 Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: Iface Name: default Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: FIRMWARE: Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: No targets found. Nov 21 15:51:58.035025 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:58.035415 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 21 15:51:58.035415 np0005531460.novalocal cinder-backup[107086]: .*?^DiscoveryAddress: 38.102.83.94,3260.*? Nov 21 15:51:58.035415 np0005531460.novalocal cinder-backup[107086]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107086) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 21 15:51:58.036612 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06', 0)] {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 21 15:51:58.037424 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.046171 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.046402 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[732d1cdf-cf52-48d4-afc2-642347664269]: (4, ('38.102.83.94:3260,4294967295 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.047646 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.058141 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.058402 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[a7aa4401-78fd-43dc-b1a1-641ffef31af8]: (4, ('tcp: [32] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 (non-flash)\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.058804 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('-m', 'session'): stdout=tcp: [32] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 (non-flash) Nov 21 15:51:58.058804 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 21 15:51:58.059078 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsi session list stdout=tcp: [32] 38.102.83.94:3260,1 iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 (non-flash) Nov 21 15:51:58.059078 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 21 15:51:58.059986 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Resulting device map defaultdict(. at 0x7fddb5210c20>, {('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06'): ({'sda'}, set())}) {{(pid=107086) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 21 15:51:58.060364 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Removing single pathed devices sda {{(pid=107086) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 21 15:51:58.061154 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.067169 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[4747c2e0-d25d-4689-8e35-71e8424e88a5]: [Errno 2] No such file or directory: 'multipathd' {{(pid=142847) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: Traceback (most recent call last): Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: ret = func(*f_args, **f_kwargs) Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: return func(*args, **kwargs) Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: return putils.execute(on_execute=on_execute, Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: obj = subprocess.Popen(cmd, Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: ^^^^^^^^^^^^^^^^^^^^^ Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 21 15:51:58.068235 np0005531460.novalocal cinder-backup[107086]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 21 15:51:58.069136 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[4747c2e0-d25d-4689-8e35-71e8424e88a5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.069708 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Flushing IO for device /dev/sda {{(pid=107086) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 21 15:51:58.070653 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.078041 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.007s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.078202 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[492dbff5-de3c-40d3-a49b-9195748753d4]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.078597 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107086) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 21 15:51:58.079400 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.127418 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.048s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.127663 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[22197617-046d-4f4e-93dc-0a6056f3fad6]: (4, ('1', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.128214 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 21 15:51:58.128505 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.linuxscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] SCSI volumes sda have been removed. {{(pid=107086) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 21 15:51:58.128952 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Disconnecting from: [('38.102.83.94:3260', 'iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06')] {{(pid=107086) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 21 15:51:58.129916 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.startup -v manual {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.141161 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op update -n node.startup -v manual" returned: 0 in 0.011s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.141322 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[13f228d4-446f-4f3e-835e-1a8196b6455a]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.141742 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:58.142586 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --logout {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.173229 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --logout" returned: 0 in 0.030s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.173611 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[498ad4f3-b0cb-45dc-98d5-8f46021b976f]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06, portal: 38.102.83.94,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06, portal: 38.102.83.94,3260] successful.\n', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.174245 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06, portal: 38.102.83.94,3260] Nov 21 15:51:58.174245 np0005531460.novalocal cinder-backup[107086]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06, portal: 38.102.83.94,3260] successful. Nov 21 15:51:58.174245 np0005531460.novalocal cinder-backup[107086]: stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:58.175335 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op delete {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 21 15:51:58.185383 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-3cd73553-e400-4d34-9879-a5a0d634cd06 -p 38.102.83.94:3260 --op delete" returned: 0 in 0.010s {{(pid=142847) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 21 15:51:58.185526 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo.privsep.daemon [-] privsep: reply[f15ced35-eb72-497e-9d27-80fbb6ab31e7]: (4, ('', '')) {{(pid=142847) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 21 15:51:58.185970 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107086) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 21 15:51:58.186408 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.base [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.164s {{(pid=107086) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 21 15:51:58.186677 np0005531460.novalocal cinder-backup[107086]: DEBUG os_brick.initiator.connectors.iscsi [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] <== disconnect_volume: return (164ms) None {{(pid=107086) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 21 15:52:01.741351 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-dec02e91-a415-4f4a-aede-ed4d1c7bb870 tempest-VolumesBackupsV39Test-394825110 None] Create backup finished. backup: 6351a90a-2902-461c-96c0-a2f99d513954. Nov 21 15:52:02.387966 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] Delete backup started, backup: 6351a90a-2902-461c-96c0-a2f99d513954. Nov 21 15:52:02.390322 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] Using swift URL https://38.102.83.94:8080/v1/AUTH_aded80cc887d4fdc84d1c8adc45b82e7 {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 21 15:52:02.390322 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.drivers.swift [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] Connect to https://38.102.83.94:8080/v1/AUTH_ in "per_user" mode {{(pid=107086) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 21 15:52:02.390510 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] delete started, backup: 6351a90a-2902-461c-96c0-a2f99d513954, container: tempest-volumesbackupsv39test-backup-container-2111697882, prefix: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 21 15:52:02.455747 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] generated object list: ['volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00001', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00002', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00003', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00004', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00005', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00006', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00007', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00008', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00009', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00010', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00011', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00012', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00013', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00014', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00015', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00016', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00017', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00018', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00019', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00020', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00021', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954_metadata', 'volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954_sha256file']. {{(pid=107086) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 21 15:52:02.484824 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00001 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.513358 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00002 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.543513 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00003 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.572598 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00004 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.601645 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00005 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.629245 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00006 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.660310 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00007 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.690006 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00008 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.715661 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00009 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.739554 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00010 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.767097 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00011 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.794096 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00012 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.826543 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00013 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.851393 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00014 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.904955 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00015 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.931930 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00016 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.961860 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00017 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:02.988712 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00018 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:03.016152 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00019 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:03.041103 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00020 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:03.062270 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954-00021 in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:03.090362 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954_metadata in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:03.117807 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] deleted object: volume_3cd73553-e400-4d34-9879-a5a0d634cd06/20251121205147/az_nova_backup_6351a90a-2902-461c-96c0-a2f99d513954_sha256file in container: tempest-volumesbackupsv39test-backup-container-2111697882. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 21 15:52:03.118561 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.backup.chunkeddriver [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] delete 6351a90a-2902-461c-96c0-a2f99d513954 finished. {{(pid=107086) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 21 15:52:03.122128 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_db.sqlalchemy.engines [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107086) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 21 15:52:03.149388 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.quota [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] Created reservations ['012ed74a-7fc4-426d-bf61-09e674855ea3', 'cc5bc363-4aee-4224-9cf2-da06a0ff32b0'] {{(pid=107086) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 21 15:52:03.179519 np0005531460.novalocal cinder-backup[107086]: INFO cinder.backup.manager [None req-9813d0b9-e8d0-4594-8ff9-7e42cdbf0e7a tempest-VolumesBackupsV39Test-394825110 None] Delete backup finished, backup 6351a90a-2902-461c-96c0-a2f99d513954 deleted. Nov 21 15:52:19.490706 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=39,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 21 15:52:46.496750 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:52:46.497167 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:53:46.501973 np0005531460.novalocal cinder-backup[107086]: DEBUG oslo_service.periodic_task [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107086) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 21 15:53:46.502519 np0005531460.novalocal cinder-backup[107086]: DEBUG cinder.manager [None req-644a5772-ac61-4ef7-bf4f-c592c4b27d0b None None] Notifying Schedulers of capabilities ... {{(pid=107086) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 21 15:54:19.494395 np0005531460.novalocal cinder-backup[107086]: DEBUG dbcounter [-] [107086] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107086) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}}