Nov 25 13:01:25.465938 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:01:34.503199 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:01:34.503644 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:02:34.504281 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:02:34.504281 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:03:25.470382 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:03:34.502993 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:03:34.503284 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:04:34.508714 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:04:34.508714 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:05:25.485021 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:05:34.513246 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:05:34.514205 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:06:34.519034 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:06:34.519443 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:07:25.477061 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:07:34.525085 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:07:34.525574 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:08:34.531875 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:08:34.531875 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:09:25.482351 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:09:34.535670 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:09:34.536232 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:10:34.535374 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:10:34.535893 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:11:25.487766 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:11:34.540998 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:11:34.542031 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:13:25.490137 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:13:34.544150 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:13:34.544648 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:14:34.547456 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:14:34.547456 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:15:06.717235 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Create backup started, backup: 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe volume: 16946f1a-76b8-4912-a33d-0d1021858840 snapshot: b82ec413-29d0-4497-aaa7-baeef082e692. Nov 25 13:15:06.721890 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumessnapshottestjson-backup-container-1316716783',created_at=2025-11-25T18:15:06Z,data_timestamp=2025-11-25T18:15:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-628971887',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='45e76a628b4447cb8b26093f8e853348',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id='b82ec413-29d0-4497-aaa7-baeef082e692',status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='60c6ae29249d495986cc475bf1f4dab5',volume_id=16946f1a-76b8-4912-a33d-0d1021858840) Nov 25 13:15:08.613054 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_45e76a628b4447cb8b26093f8e853348 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:15:08.613495 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:15:08.614558 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:15:08.616805 np0005535648.novalocal cinder-backup[107475]: INFO oslo.privsep.daemon [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Running privsep helper: ['sudo', 'cinder-rootwrap', '/etc/cinder/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/cinder/cinder.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmplnvoi5jv/privsep.sock'] Nov 25 13:15:08.635010 np0005535648.novalocal sudo[140710]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmplnvoi5jv/privsep.sock Nov 25 13:15:09.437071 np0005535648.novalocal cinder-backup[107475]: INFO oslo.privsep.daemon [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Spawned new privsep daemon via rootwrap Nov 25 13:15:09.437376 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Accepted privsep connection to /tmp/tmplnvoi5jv/privsep.sock {{(pid=107475) __init__ /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:377}} Nov 25 13:15:09.514394 np0005535648.novalocal cinder-backup[107475]: INFO oslo.privsep.daemon [-] privsep daemon starting Nov 25 13:15:09.514521 np0005535648.novalocal cinder-backup[107475]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 Nov 25 13:15:09.514521 np0005535648.novalocal cinder-backup[107475]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_READ_SEARCH|CAP_SYS_ADMIN/CAP_DAC_READ_SEARCH|CAP_SYS_ADMIN/none Nov 25 13:15:09.514772 np0005535648.novalocal cinder-backup[107475]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 140715 Nov 25 13:15:09.516138 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[376e2a8d-b344-413d-a0a5-2b2bf3923d66]: (2,) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:09.620594 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:09.627389 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:09.627574 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b620af47-450d-4d14-a5b9-fedcd35c80f9]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[decc616f-8600-475b-a008-8b734d1dfd43]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:09.631910 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:15:09.632434 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[decc616f-8600-475b-a008-8b734d1dfd43]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:09.632662 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:15:09.633034 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:15:09.633322 np0005535648.novalocal cinder-backup[107475]: WARNING os_brick.initiator.connectors.nvmeof [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath' Nov 25 13:15:09.633891 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:09.642359 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:09.642503 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7abe6abd-334e-4f16-9577-464b14d5fa7b]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:09.643208 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:09.652526 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:09.652842 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3b3ce40-b67d-4aef-9e3d-0caf466c0221]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:09.654283 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[00866bae-1eb6-46d8-b046-d0f4ea5b7cfd]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:09.654886 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:09.664047 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:15:09.664345 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:15:09.666445 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:15:09.668103 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fe15:5911', 'fe80::49e:61ff:fe37:ac43', 'fe80::fc16:3eff:fe1b:80d0', 'fe80::9c77:87ff:fe31:6a9c', 'fe80::fc16:3eff:fed2:1637'] Nov 25 13:15:09.668411 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:15:09.668641 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:15:09.669182 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] <== get_connector_properties: return (1054ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fe15:5911', 'fe80::49e:61ff:fe37:ac43', 'fe80::fc16:3eff:fe1b:80d0', 'fe80::9c77:87ff:fe31:6a9c', 'fe80::fc16:3eff:fed2:1637']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:15:13.192247 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:15:13.193962 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292', 'target_lun': 0, 'volume_id': 'c64faead-1248-4645-aff9-818c115be292', 'auth_method': 'CHAP', 'auth_username': 'DCE6d5LSAxL6msUzfHiN', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:15:13.194357 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:15:13.195112 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:15:13.195450 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292', 'target_lun': 0, 'volume_id': 'c64faead-1248-4645-aff9-818c115be292', 'auth_method': 'CHAP', 'auth_username': 'DCE6d5LSAxL6msUzfHiN', 'auth_password': 'FB3P3u7U6HzSMgHd', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:15:13.196579 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:13.202826 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0eb0ee04-c644-42f6-9329-f7a0e883c9cb]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:15:13.207489 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:15:13.208223 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0eb0ee04-c644-42f6-9329-f7a0e883c9cb]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:13.209059 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:15:13.209712 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:15:13.210092 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:15:13.210912 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:13.221905 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260" returned: 21 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:13.222117 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e7bb87d-199b-4463-8ff0-5e4f9d4bb5c2]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:13.222609 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:15:13.222609 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:13.223655 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:13.236822 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:13.237125 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec71dc01-3cf2-4784-a9b0-d7c42680a29e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:13.237687 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292] added Nov 25 13:15:13.237687 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:15:13.237687 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:15:13.237687 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:15:13.237687 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:13.238177 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:15:13.239258 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.029(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:15:13.239733 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:15:14.241497 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:15:14.243095 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.255041 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.255458 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0022870c-fa3c-409c-be0d-785bd8a63431]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:15:14.257409 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.260317 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.271233 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.271470 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf5a628f-0db7-4148-9480-3af012fb2bf9]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.272196 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.273272 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.283867 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.284195 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1229a21-6449-49e8-a72f-0c4970dbb4d6]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.284827 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.286314 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v DCE6d5LSAxL6msUzfHiN {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.299316 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v DCE6d5LSAxL6msUzfHiN" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.299583 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0053b4bd-02fd-4512-96f6-8497407f65b0]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.300124 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'DCE6d5LSAxL6msUzfHiN'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.302111 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.316702 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.317067 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e72ce963-22ea-4ea7-8ba6-03831e39cf62]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.317828 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.319435 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.331980 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.332723 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0b0dcf7-fee8-4c7b-98ab-2b058d4abbb6]: (4, ('tcp: [14] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-3f559a77-0b60-4ae4-9cad-6788ebaa5e69 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.333207 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('-m', 'session'): stdout=tcp: [14] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-3f559a77-0b60-4ae4-9cad-6788ebaa5e69 (non-flash) Nov 25 13:15:14.333207 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:15:14.333467 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsi session list stdout=tcp: [14] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-3f559a77-0b60-4ae4-9cad-6788ebaa5e69 (non-flash) Nov 25 13:15:14.333467 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:15:14.335957 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.355071 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --login" returned: 0 in 0.019s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.355165 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb995272-99fa-405d-83fd-caa713488cc4]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.355802 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292, portal: 38.102.83.58,3260] successful. Nov 25 13:15:14.355802 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.357348 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.369853 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.370034 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ae66db3-4414-4fff-a973-aec461212fc4]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.370351 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:14.371751 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.388773 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.389616 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[04299478-7257-41ef-8fbf-1e7fb418cfd7]: (4, ('tcp: [14] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-3f559a77-0b60-4ae4-9cad-6788ebaa5e69 (non-flash)\ntcp: [15] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.390060 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('-m', 'session'): stdout=tcp: [14] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-3f559a77-0b60-4ae4-9cad-6788ebaa5e69 (non-flash) Nov 25 13:15:14.390060 np0005535648.novalocal cinder-backup[107475]: tcp: [15] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 (non-flash) Nov 25 13:15:14.390060 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:15:14.390235 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsi session list stdout=tcp: [14] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-3f559a77-0b60-4ae4-9cad-6788ebaa5e69 (non-flash) Nov 25 13:15:14.390235 np0005535648.novalocal cinder-backup[107475]: tcp: [15] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 (non-flash) Nov 25 13:15:14.390235 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:15:14.390698 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.181s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:15:14.390881 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:15:14.391997 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] HCTL ('3', '-', '-', 0) found on session 15 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:15:14.392161 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:15:14.394798 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:14.422557 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.029s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:14.422868 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[35b2f5b8-6153-4483-b0ba-361bbe6e94c3]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:14.424338 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Searching for a device in session 15 and hctl ['3', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:15:15.426584 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Searching for a device in session 15 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:15:15.427432 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292', 'target_lun': 0, 'volume_id': 'c64faead-1248-4645-aff9-818c115be292', 'auth_method': 'CHAP', 'auth_username': 'DCE6d5LSAxL6msUzfHiN', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:15:15.428489 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.233s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:15:15.428964 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] <== connect_volume: return (2235ms) {'type': 'block', 'scsi_wwn': '360014058ba26530e51b4f9eb055e0fe1', 'path': '/dev/sdb'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:15:15.430011 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:15.452965 np0005535648.novalocal sudo[140882]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 25 13:15:15.487732 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-2354c152-d9b6-43d1-afd9-8b8b45ad6795 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:15:15.761836 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.332s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:15.777354 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] _create_container started, container: tempest-volumessnapshottestjson-backup-container-1316716783,backup: 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:15:15.878864 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] generate_object_name_prefix: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:15:15.884903 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] starting backup of volume: 16946f1a-76b8-4912-a33d-0d1021858840, volume size: 1073741824, object names prefix volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:15:16.289624 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:16.419465 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:16.419799 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:16.537676 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:16.538244 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:16.885425 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:17.010231 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:17.010604 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:17.143479 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:17.143894 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:17.262312 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:17.389961 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:17.390059 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:17.653754 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:17.654078 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:17.823502 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:17.933137 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:17.933420 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:18.087142 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:18.087270 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:18.216452 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:18.316660 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:18.316974 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:18.470669 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:18.470883 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:18.594731 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:18.752184 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:18.752591 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:18.866425 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:18.866559 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:19.016579 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:19.146191 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:19.146336 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:19.289588 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:19.289730 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:19.889612 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:20.046532 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:20.046731 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:20.192706 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:20.192917 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:20.570890 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:20.713764 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:20.713902 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:20.873895 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:20.874288 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:21.027141 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:21.141888 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:21.142027 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:21.295655 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:21.295980 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:21.546833 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:21.654726 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:21.654900 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:21.773874 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:21.773874 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:21.914174 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:22.043597 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:22.043789 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:22.182346 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:22.182511 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:22.320776 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:22.473812 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:22.473927 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:22.576399 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:22.576722 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:22.739835 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:22.858393 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:22.858393 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:23.016910 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:23.017073 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:23.206617 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:23.307942 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:23.308141 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:23.432692 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:23.432833 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:23.579849 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:23.701666 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:23.701751 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:23.818950 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:23.819060 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:24.181960 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:24.321627 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:24.322037 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:24.466286 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:24.466601 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:24.780095 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:24.878826 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:24.878993 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:25.031047 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:25.031145 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:25.397446 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:25.495891 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:25.495891 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:25.641608 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:25.641608 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:25.802855 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:25.908740 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:25.908740 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:26.182110 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:26.182261 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:26.280452 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:15:26.376018 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:15:26.376311 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:15:26.476082 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup MD5 for volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:15:26.476562 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:15:26.481045 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:15:26.493355 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:15:26.493701 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:15:26.504894 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:15:26.505174 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:15:26.514690 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:15:26.515002 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] _write_sha256file started, container name: tempest-volumessnapshottestjson-backup-container-1316716783, sha256file filename: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:15:26.614855 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:15:26.615812 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] _write_metadata started, container name: tempest-volumessnapshottestjson-backup-container-1316716783, metadata filename: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:15:26.656875 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe",\n "backup_name": "tempest-VolumesSnapshotTestJSON-Backup-628971887",\n "created_at": "2025-11-25 18:15:06+00:00",\n "objects": [\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "16946f1a-76b8-4912-a33d-0d1021858840",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"16946f1a-76b8-4912-a33d-0d1021858840\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"60c6ae29249d495986cc475bf1f4dab5\\", \\"project_id\\": \\"45e76a628b4447cb8b26093f8e853348\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"available\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:15:04.000000\\", \\"launched_at\\": \\"2025-11-25T18:15:04.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesSnapshotTestJSON-Volume-1586922092\\", \\"display_description\\": null, \\"provider_location\\": null, \\"provider_auth\\": null, \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": null, \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:15:04.000000\\", \\"updated_at\\": \\"2025-11-25T18:15:04.000000\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:15:26.665456 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] backup 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:15:26.771515 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:26.794596 np0005535648.novalocal sudo[141134]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 25 13:15:27.194380 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.423s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.196409 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292', 'target_lun': 0, 'volume_id': 'c64faead-1248-4645-aff9-818c115be292', 'auth_method': 'CHAP', 'auth_username': 'DCE6d5LSAxL6msUzfHiN', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014058ba26530e51b4f9eb055e0fe1', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:15:27.196851 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:15:27.197888 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:15:27.200272 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.207833 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.208256 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0141cef-0fac-4ae2-b496-df9b248c5e64]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:15:27.208655 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:15:27.208984 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:15:27.208984 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:15:27.208984 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:15:27.209910 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:15:27.212430 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.221812 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.222080 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1e937ca-10d4-42ad-908e-e259fb00c10c]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.223215 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.265201 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.042s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.265447 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[43d80d80-af3e-438e-9898-b16f7d7a2dbc]: (4, ('tcp: [15] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.266022 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('-m', 'session'): stdout=tcp: [15] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 (non-flash) Nov 25 13:15:27.266022 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:15:27.266298 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsi session list stdout=tcp: [15] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 (non-flash) Nov 25 13:15:27.266298 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:15:27.267988 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Resulting device map defaultdict(. at 0x7f9bd9a419e0>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292'): ({'sdb'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:15:27.268580 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Removing single pathed devices sdb {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:15:27.269708 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.276139 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[25b5b80c-cba7-43b1-a36c-13d3a70cf0c8]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:15:27.277311 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:15:27.278359 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[25b5b80c-cba7-43b1-a36c-13d3a70cf0c8]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.278932 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Flushing IO for device /dev/sdb {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:15:27.280001 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.287953 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.288618 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[831380af-7afb-411d-a0a5-5bd7123a378e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.288881 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:15:27.289930 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.339654 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.049s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.339889 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5f0bf97-7058-41a7-a527-48eb9ed81563]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.340407 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:15:27.340745 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:15:27.341193 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:15:27.342580 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.354034 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.354133 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9931a7b-a225-40b4-8ca7-e30954f45a08]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.354476 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:27.355901 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.387284 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --logout" returned: 0 in 0.029s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.387284 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c604d4f-66b4-4016-91b4-81bf8ff35d30]: (4, ('Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292, portal: 38.102.83.58,3260]\nLogout of [sid: 15, target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.387609 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292, portal: 38.102.83.58,3260] Nov 25 13:15:27.387609 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 15, target: iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292, portal: 38.102.83.58,3260] successful. Nov 25 13:15:27.387609 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:27.388807 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:15:27.398995 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c64faead-1248-4645-aff9-818c115be292 -p 38.102.83.58:3260 --op delete" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:15:27.399293 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4d8290b-e6c6-4a01-9d9b-e06da7c0edbb]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:15:27.399661 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:15:27.399966 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.202s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:15:27.400162 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] <== disconnect_volume: return (203ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:15:30.983017 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-67ffcbe0-3db1-4cd0-8230-097e3140147a tempest-VolumesSnapshotTestJSON-2115482626 None] Create backup finished. backup: 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe. Nov 25 13:15:31.800675 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] Delete backup started, backup: 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe. Nov 25 13:15:31.801488 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_45e76a628b4447cb8b26093f8e853348 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:15:31.801757 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:15:31.801757 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] delete started, backup: 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe, container: tempest-volumessnapshottestjson-backup-container-1316716783, prefix: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:15:31.888943 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] generated object list: ['volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00001', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00002', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00003', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00004', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00005', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00006', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00007', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00008', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00009', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00010', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00011', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00012', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00013', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00014', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00015', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00016', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00017', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00018', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00019', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00020', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00021', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe_metadata', 'volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:15:31.932776 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00001 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:31.978330 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00002 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.018462 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00003 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.064277 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00004 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.104594 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00005 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.152515 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00006 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.192367 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00007 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.235171 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00008 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.286317 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00009 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.325680 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00010 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.371752 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00011 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.407193 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00012 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.436723 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00013 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.467971 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00014 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.496326 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00015 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.523673 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00016 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.549146 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00017 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.578036 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00018 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.600950 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00019 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.646624 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00020 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.680463 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe-00021 in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.708444 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe_metadata in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.741602 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] deleted object: volume_16946f1a-76b8-4912-a33d-0d1021858840/20251125181515/az_nova_backup_8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe_sha256file in container: tempest-volumessnapshottestjson-backup-container-1316716783. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:15:32.741986 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] delete 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:15:32.746760 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:15:32.790573 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] Created reservations ['c71fb255-4e70-48b5-a328-4a96ead79880', 'a7008cff-d1df-4949-8e78-7f5af70bf81e'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:15:32.823124 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-b4a2253b-a028-438e-90ed-766f171ad2cf tempest-VolumesSnapshotTestJSON-2115482626 None] Delete backup finished, backup 8f30e44d-b9df-4e1d-bcc8-e43ef7677cbe deleted. Nov 25 13:15:34.543648 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:15:34.543648 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:15:42.821440 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=42,cinder:UPDATE=13,cinder:INSERT=2 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:16:34.547219 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:16:34.547748 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:17:25.497133 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:17:34.547503 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:17:34.548079 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:18:34.553170 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:18:34.553700 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:19:25.498808 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:19:34.559209 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:19:34.559730 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:20:34.565596 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:20:34.565596 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:21:25.501769 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:21:34.571106 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:21:34.572031 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:22:34.571839 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:22:34.572263 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:23:25.499574 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:23:34.578258 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:23:34.578788 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:24:34.584559 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:24:34.584977 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:25:25.501100 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:25:34.589088 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:25:34.589593 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:26:34.594562 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:26:34.595095 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:27:25.500644 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:27:34.598643 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:27:34.599170 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:28:34.604602 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:28:34.604602 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:29:25.502373 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:29:34.609656 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:29:34.610227 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:30:34.615016 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:30:34.616101 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:31:25.503664 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:31:34.617480 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:31:34.618342 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:32:33.093825 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Create backup started, backup: 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0 volume: 67c94551-2d2b-4e10-a520-2c3d119a7cf8. Nov 25 13:32:33.098634 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-139093474',created_at=2025-11-25T18:32:33Z,data_timestamp=2025-11-25T18:32:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-984367673',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='8ad212ee97684fec9bf2f2dc25f47549',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='39069811c8de4688bdc78baf4cf5dc3b',volume_id=67c94551-2d2b-4e10-a520-2c3d119a7cf8) Nov 25 13:32:34.613344 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:32:34.613773 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:32:35.333287 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:32:35.333411 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:32:35.333715 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:32:35.334685 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:35.341964 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:35.342229 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2335d017-7f79-44c4-b8ba-8ec8b4f6eb3c]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f8dc888f-92df-4398-8401-8ad141d07c31]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:35.343853 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:32:35.344437 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8dc888f-92df-4398-8401-8ad141d07c31]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:35.344594 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:32:35.344874 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:32:35.345381 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:35.354437 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:35.354796 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cf34496-7125-49c1-a988-f79802424c71]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:35.355470 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:35.365479 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:35.365905 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0edf2a36-9cbe-4415-a628-13b3981bfcb7]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:35.366728 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[52e1065b-0312-4559-bbb4-dd064abcd10b]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:35.367027 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:35.376434 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:32:35.376784 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:32:35.378436 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:32:35.379213 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fec9:5ebd', 'fe80::e8d8:e5ff:fe05:ec73'] Nov 25 13:32:35.381319 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:32:35.381319 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:32:35.381319 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (45ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fec9:5ebd', 'fe80::e8d8:e5ff:fe05:ec73']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:32:38.888843 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:32:38.890381 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'target_lun': 0, 'volume_id': '8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'auth_method': 'CHAP', 'auth_username': 'HjR5HKvipk8HvxyHEoEj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:32:38.890931 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:32:38.891690 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:32:38.892035 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'target_lun': 0, 'volume_id': '8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'auth_method': 'CHAP', 'auth_username': 'HjR5HKvipk8HvxyHEoEj', 'auth_password': 'Ai2mmMRNmrtpRLp9', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:32:38.894346 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:38.900624 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ce7e4afa-22c7-4357-9857-a6ca07d958ed]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:32:38.903614 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:32:38.905914 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce7e4afa-22c7-4357-9857-a6ca07d958ed]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:38.906992 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:32:38.908049 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:32:38.908718 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:32:38.910094 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:38.920823 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260" returned: 21 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:38.921044 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3a51529-9ee3-465d-b649-2401dbe9480e]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:38.921595 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:32:38.921595 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:38.922633 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:38.933237 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:38.933640 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[35c17bbe-1af3-43f0-9d0e-02a87afc627a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:38.934181 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e] added Nov 25 13:32:38.934181 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:32:38.934181 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:32:38.934181 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:32:38.934181 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:38.934455 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:32:38.935220 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.027(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:32:38.935578 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:32:39.937584 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:32:39.938632 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:39.950912 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:39.951626 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[433f661d-d000-4f82-920e-c1cadf3252c1]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:32:39.954101 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:39.957622 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:39.963824 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:39.964025 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0881b34-6e4e-424e-af87-b642e520f5ed]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:39.964513 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:39.965585 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:39.973169 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:39.973497 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a977a58c-b784-40f2-8df3-b4032772cea8]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:39.973947 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:39.974886 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.auth.username -v HjR5HKvipk8HvxyHEoEj {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:39.985835 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.auth.username -v HjR5HKvipk8HvxyHEoEj" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:39.986173 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd983aab-7fbb-40ec-ad1f-cbc8c0c42b24]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:39.986651 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'HjR5HKvipk8HvxyHEoEj'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:39.987829 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:39.998916 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:39.999497 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc6b4368-1943-4a24-9d78-6a97b0ab2d2f]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:40.000110 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:40.001096 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:40.013527 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:40.013873 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbb9213f-50d2-4bbb-861b-42ded048bcaf]: (4, ('tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:40.014287 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash) Nov 25 13:32:40.014287 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:32:40.014590 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash) Nov 25 13:32:40.014590 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:32:40.015617 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:40.038163 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --login" returned: 0 in 0.022s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:40.038589 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[389891e9-527a-4ecf-8efc-eb6fbb90f4ce]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:40.039405 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e, portal: 38.102.83.58,3260] successful. Nov 25 13:32:40.039405 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:40.040743 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:40.054141 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:40.054479 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a9d487ad-2314-462a-9b8d-831008660478]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:40.054848 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:40.055693 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:40.068480 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:40.068746 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c02f2961-4348-45ab-81cb-f1dcc6966eba]: (4, ('tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash)\ntcp: [23] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:40.069253 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash) Nov 25 13:32:40.069253 np0005535648.novalocal cinder-backup[107475]: tcp: [23] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e (non-flash) Nov 25 13:32:40.069253 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:32:40.069464 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash) Nov 25 13:32:40.069464 np0005535648.novalocal cinder-backup[107475]: tcp: [23] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e (non-flash) Nov 25 13:32:40.069464 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:32:40.069940 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.162s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:32:40.070153 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:32:40.070891 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] HCTL ('3', '-', '-', 0) found on session 23 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:32:40.071066 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:32:40.071994 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:40.098487 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.025s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:40.098487 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e8abad3-79ac-4fff-adf6-42444dd438d8]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:40.099044 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 23 and hctl ['3', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:32:41.101865 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 23 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:32:41.102765 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'target_lun': 0, 'volume_id': '8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'auth_method': 'CHAP', 'auth_username': 'HjR5HKvipk8HvxyHEoEj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:32:41.103834 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.212s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:32:41.104166 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2213ms) {'type': 'block', 'scsi_wwn': '360014051fb3c5b0e3484c5a8e0bec624', 'path': '/dev/sdb'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:32:41.104897 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:41.134066 np0005535648.novalocal sudo[149078]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 25 13:32:41.478680 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.373s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:41.487307 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:32:41.497747 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] _create_container started, container: tempest-volumesbackupstest-backup-container-139093474,backup: 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:32:41.692163 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] generate_object_name_prefix: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0 {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:32:41.697213 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] starting backup of volume: 67c94551-2d2b-4e10-a520-2c3d119a7cf8, volume size: 1073741824, object names prefix volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:32:41.966409 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:42.106803 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:42.107474 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:42.269520 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:42.269520 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:42.507725 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:42.617193 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:42.617430 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:42.777051 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:42.777270 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:42.964100 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:43.067070 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:43.067290 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:43.257647 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:43.258302 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:43.412431 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:43.509778 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:43.509885 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:43.670575 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:43.670731 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:44.079286 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:44.214684 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:44.214684 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:44.376642 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:44.376985 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:44.825463 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:44.963736 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:44.963996 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:45.115929 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:45.115929 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:45.335353 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:45.500132 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:45.500132 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:45.647347 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:45.647665 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:45.826448 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:45.958283 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:45.958688 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:46.084216 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:46.084461 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:46.266333 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:46.370828 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:46.370959 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:46.495023 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:46.495206 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:46.671831 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:46.832210 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:46.832345 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:46.980057 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:46.980288 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:47.174569 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:47.295502 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:47.295648 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:47.417910 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:47.418093 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:47.556642 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:47.657472 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:47.657704 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:47.798323 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:47.798958 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:48.176774 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:48.304726 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:48.304868 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:48.449147 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:48.449322 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:48.633237 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:48.737794 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:48.737794 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:48.846034 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:48.846707 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:49.290009 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:49.433521 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:49.433697 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:49.595294 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:49.595294 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:50.089786 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:50.210058 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:50.210303 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:50.360766 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:50.360963 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:50.731289 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:50.835752 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:50.835894 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:50.997790 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:50.998661 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:51.391104 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:51.521194 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:51.521433 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:51.671829 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:51.672129 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:51.841299 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:51.975861 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:51.976028 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:52.087965 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:52.088122 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:52.253986 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:52.387311 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:52.387424 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:52.540494 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:52.541079 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:52.882761 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:32:52.961790 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:32:52.961920 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:32:53.067936 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:32:53.068093 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:32:53.073942 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:32:53.083780 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:32:53.084067 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:32:53.088531 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:32:53.088824 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:32:53.093269 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:32:53.093760 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-139093474, sha256file filename: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:32:53.181732 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:32:53.182668 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-139093474, metadata filename: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:32:53.217256 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0",\n "backup_name": "tempest-VolumesBackupsTest-Backup-984367673",\n "created_at": "2025-11-25 18:32:33+00:00",\n "objects": [\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "67c94551-2d2b-4e10-a520-2c3d119a7cf8",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"67c94551-2d2b-4e10-a520-2c3d119a7cf8\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"39069811c8de4688bdc78baf4cf5dc3b\\", \\"project_id\\": \\"8ad212ee97684fec9bf2f2dc25f47549\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:31:44.000000\\", \\"launched_at\\": \\"2025-11-25T18:31:44.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1368736163\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 0\\", \\"provider_auth\\": \\"CHAP hLEgFee5LLsHgsKFVGKc uvJ8v3BqoGPdf4R7\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:31:44.000000\\", \\"updated_at\\": \\"2025-11-25T18:32:33.000000\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:32:53.223324 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] backup 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0 finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:32:53.319684 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.339567 np0005535648.novalocal sudo[149125]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 25 13:32:53.677922 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.358s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.679456 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'target_lun': 0, 'volume_id': '8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 'auth_method': 'CHAP', 'auth_username': 'HjR5HKvipk8HvxyHEoEj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014051fb3c5b0e3484c5a8e0bec624', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:32:53.679832 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:32:53.680444 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:32:53.681959 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.691731 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.691960 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5b36965-5127-45fe-ad14-973ea128ccfe]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:32:53.692399 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:32:53.692811 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:32:53.692811 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:32:53.692811 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:32:53.693170 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:32:53.693816 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.702128 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.702349 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab15b413-5776-4c04-b23d-c8bcc24a1460]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8\n38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.703075 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.715339 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.715680 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[47fd467b-2d29-4fdf-aaf6-405c46b687e0]: (4, ('tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash)\ntcp: [23] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.716139 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash) Nov 25 13:32:53.716139 np0005535648.novalocal cinder-backup[107475]: tcp: [23] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e (non-flash) Nov 25 13:32:53.716139 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:32:53.716418 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [22] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-67c94551-2d2b-4e10-a520-2c3d119a7cf8 (non-flash) Nov 25 13:32:53.716418 np0005535648.novalocal cinder-backup[107475]: tcp: [23] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e (non-flash) Nov 25 13:32:53.716418 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:32:53.717390 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bdc9df380>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e'): ({'sdb'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:32:53.717850 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sdb {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:32:53.718501 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.724643 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f374e752-6f17-4e90-a997-caa511ff156c]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:32:53.725608 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:32:53.726373 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f374e752-6f17-4e90-a997-caa511ff156c]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.727027 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sdb {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:32:53.727694 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.733915 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.006s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.734164 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[67efcc1b-62dd-4447-a416-ebf27c7924bc]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.734642 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:32:53.735277 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.797632 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.062s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.797901 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fee9cf3c-30f6-4032-bf03-71e9adb49070]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.798405 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:32:53.798741 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:32:53.799330 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:32:53.800425 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.812346 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.812583 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c17ca53-0e98-461a-b31b-758ace531295]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.813075 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:53.813942 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.846714 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --logout" returned: 0 in 0.032s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.847022 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[761863d4-882b-42b3-a9f4-072e537b5f2f]: (4, ('Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e, portal: 38.102.83.58,3260]\nLogout of [sid: 23, target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.847575 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e, portal: 38.102.83.58,3260] Nov 25 13:32:53.847575 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 23, target: iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e, portal: 38.102.83.58,3260] successful. Nov 25 13:32:53.847575 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:53.849042 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:32:53.862132 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-8e9a98dc-d66c-49bd-b3d1-4362824ec53e -p 38.102.83.58:3260 --op delete" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:32:53.862350 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f5c7e01-d152-4524-9154-cdaf242ff1aa]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:32:53.862788 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:32:53.863204 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.183s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:32:53.863481 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (183ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:32:57.405571 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-1fbdc2b2-df63-411e-a6b2-fdb64a72ade6 tempest-VolumesBackupsTest-210128247 None] Create backup finished. backup: 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0. Nov 25 13:32:57.842609 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] Delete backup started, backup: 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0. Nov 25 13:32:57.843806 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:32:57.844207 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:32:57.844681 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] delete started, backup: 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0, container: tempest-volumesbackupstest-backup-container-139093474, prefix: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:32:57.918881 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00001', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00002', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00003', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00004', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00005', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00006', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00007', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00008', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00009', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00010', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00011', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00012', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00013', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00014', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00015', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00016', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00017', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00018', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00019', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00020', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00021', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0_metadata', 'volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:32:57.954482 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00001 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:57.984570 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00002 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.013154 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00003 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.046647 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00004 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.074710 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00005 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.105379 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00006 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.142359 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00007 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.173084 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00008 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.197416 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00009 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.223838 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00010 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.254916 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00011 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.288327 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00012 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.321102 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00013 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.353785 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00014 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.382499 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00015 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.415095 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00016 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.449160 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00017 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.477116 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00018 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.504570 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00019 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.530811 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00020 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.559737 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0-00021 in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.586922 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0_metadata in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.615378 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] deleted object: volume_67c94551-2d2b-4e10-a520-2c3d119a7cf8/20251125183241/az_nova_backup_49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0_sha256file in container: tempest-volumesbackupstest-backup-container-139093474. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:32:58.615860 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] delete 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0 finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:32:58.620197 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:32:58.641643 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] Created reservations ['04622337-943e-450d-8fde-de96b5bc5d2a', 'a4d8b63d-c3e5-45a5-9c0b-2caef4a23d5f'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:32:58.664188 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-d5bab49d-e0f1-43ee-a02d-3f7f3a33d04c tempest-VolumesBackupsTest-210128247 None] Delete backup finished, backup 49fa7a2c-2fc4-4b8e-9b2d-c6aa051396d0 deleted. Nov 25 13:33:08.663651 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=41,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:33:16.824650 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Create backup started, backup: 456c2964-56d4-4356-8ad1-d93afa40def5 volume: caad167b-7322-4d3c-918b-1ef8530687c1. Nov 25 13:33:16.829255 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1393287685',created_at=2025-11-25T18:33:16Z,data_timestamp=2025-11-25T18:33:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1647097626',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=456c2964-56d4-4356-8ad1-d93afa40def5,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='8ad212ee97684fec9bf2f2dc25f47549',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='39069811c8de4688bdc78baf4cf5dc3b',volume_id=caad167b-7322-4d3c-918b-1ef8530687c1) Nov 25 13:33:16.862271 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:33:16.862565 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:33:16.863067 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:33:16.865617 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:16.871239 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:16.871736 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[9af5bdd1-34d8-4257-94c0-bf96c2866e18]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[51a72b9c-ef8c-4700-ba5b-83382a1fe898]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:16.874317 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:33:16.876172 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[51a72b9c-ef8c-4700-ba5b-83382a1fe898]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:16.876367 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:33:16.876849 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:33:16.877855 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:16.884858 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:16.885199 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[33764c98-c1ef-423e-b38e-bb1f5dbd484c]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:16.886293 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:16.893565 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:16.893963 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5cc2795-57a9-4738-92b8-cb696cba2e25]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:16.895143 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1507ad9a-b716-4ebb-8f6b-06795e05527c]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:16.895888 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:16.908847 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:33:16.908987 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:33:16.911092 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:33:16.911973 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40'] Nov 25 13:33:16.912334 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:33:16.912645 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:33:16.913154 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (49ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:33:20.422672 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:33:20.424308 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1', 'target_lun': 0, 'volume_id': 'caad167b-7322-4d3c-918b-1ef8530687c1', 'auth_method': 'CHAP', 'auth_username': 'Yx5aT3bCRPy9n2YRWwbp', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:33:20.424769 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:33:20.425627 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:33:20.426010 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1', 'target_lun': 0, 'volume_id': 'caad167b-7322-4d3c-918b-1ef8530687c1', 'auth_method': 'CHAP', 'auth_username': 'Yx5aT3bCRPy9n2YRWwbp', 'auth_password': 'oNpP5h9jJZcvZXF4', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:33:20.427661 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:20.433368 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[cb25bcd7-6b1a-4115-890d-f04863747769]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:33:20.435480 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:33:20.437289 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb25bcd7-6b1a-4115-890d-f04863747769]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:20.438227 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:33:20.439043 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:33:20.439441 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:33:20.440525 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:20.450490 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260" returned: 21 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:20.450880 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d00293cd-f46d-4cc6-b387-7def80585528]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:20.451395 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:33:20.451395 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:20.452466 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:20.463981 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:20.464312 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbff1d7b-3627-4d6b-b221-4f10bf0f40c5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:20.464853 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1] added Nov 25 13:33:20.464853 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:33:20.464853 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:33:20.464853 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:33:20.464853 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:20.465604 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:33:20.465912 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.026(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:33:20.466287 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:33:21.468410 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:33:21.469980 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.481347 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.481816 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[624440d2-e02a-40c6-b356-5e23c68f9f08]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:33:21.483529 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.486303 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.495951 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.496279 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d68df393-31e3-4bbf-b21b-e5d9ceff6904]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.496828 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.497939 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.506075 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.506445 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[182e775b-ba6d-423b-8652-ea01280422cf]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.506998 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.508216 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v Yx5aT3bCRPy9n2YRWwbp {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.516679 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v Yx5aT3bCRPy9n2YRWwbp" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.517012 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb19d831-17b3-4d3e-89de-16c7c19312b6]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.517426 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'Yx5aT3bCRPy9n2YRWwbp'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.518525 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.526709 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.526962 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c1eff5d-1f4e-4f06-a2db-792c72163bec]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.527578 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.528423 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.535261 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.535511 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7641bfd3-4642-4478-8e6c-c6573ae15267]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.535968 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 25 13:33:21.535968 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:21.536268 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 25 13:33:21.536268 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:33:21.536684 np0005535648.novalocal cinder-backup[107475]: WARNING os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 25 13:33:21.536684 np0005535648.novalocal cinder-backup[107475]: Nov 25 13:33:21.537756 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.551092 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --login" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.552129 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[98e68440-3f9e-4e7e-86a8-6432d9df938b]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.552609 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1, portal: 38.102.83.58,3260] successful. Nov 25 13:33:21.552609 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.553516 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.565121 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.565387 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1e95ae5-6ee5-4a70-8768-352f5343b9bb]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.565873 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:21.566757 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.579224 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.579450 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd968b03-91f0-4faa-a469-8ac7d5631ea1]: (4, ('tcp: [24] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.579907 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [24] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 (non-flash) Nov 25 13:33:21.579907 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:21.580167 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [24] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 (non-flash) Nov 25 13:33:21.580167 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:33:21.580701 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.142s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:33:21.580983 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:33:21.581523 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] HCTL ('2', '-', '-', 0) found on session 24 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:33:21.581763 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:33:21.582446 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:21.605906 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.023s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:21.606073 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[44d65f38-c561-4e52-bebe-6b3cce427654]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:21.606872 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 24 and hctl ['2', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:33:22.609641 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 24 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:33:22.610041 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1', 'target_lun': 0, 'volume_id': 'caad167b-7322-4d3c-918b-1ef8530687c1', 'auth_method': 'CHAP', 'auth_username': 'Yx5aT3bCRPy9n2YRWwbp', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:33:22.610785 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.185s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:33:22.611167 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2186ms) {'type': 'block', 'scsi_wwn': '36001405e8ca4c1b423c4243b8d75eec3', 'path': '/dev/sda'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:33:22.611856 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:22.637098 np0005535648.novalocal sudo[149505]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 25 13:33:22.954395 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.342s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:22.960646 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:33:22.971817 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1393287685,backup: 456c2964-56d4-4356-8ad1-d93afa40def5. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:33:23.042927 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] generate_object_name_prefix: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5 {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:33:23.058791 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] starting backup of volume: caad167b-7322-4d3c-918b-1ef8530687c1, volume size: 1073741824, object names prefix volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:33:23.301627 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:23.844866 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 19857195 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:23.845313 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:24.152119 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001: 82571f7121cb9475563edde615e4e4df {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:24.152377 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:24.384712 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:24.491417 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:24.491973 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:24.644122 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:24.644364 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:24.870391 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:25.017252 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 1362431 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:25.017516 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:25.140561 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003: a35891500449df17e893f84bd88472fb {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:25.140904 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:25.371433 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:25.509110 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:25.509270 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:25.658280 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:25.658565 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:25.823774 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:25.956820 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:25.957256 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:26.108657 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:26.108657 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:26.278402 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:26.401622 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:26.401883 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:26.548379 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:26.548664 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:26.790229 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:26.924260 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:26.924395 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:27.051678 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:27.052270 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:27.276027 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:27.415852 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:27.416131 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:27.613104 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:27.613258 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:27.768187 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:27.886638 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:27.886792 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:28.035333 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:28.035528 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:28.210814 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:28.346103 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:28.346252 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:28.495397 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:28.495623 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:28.729818 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:28.847248 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:28.847377 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:28.965477 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:28.965659 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:29.142278 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:29.277619 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:29.278019 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:29.631722 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:29.632032 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:29.748910 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:29.887116 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:29.887325 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:30.030256 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:30.030399 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:30.431634 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:30.568063 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:30.568250 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:30.692887 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:30.693095 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:30.819015 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:30.937829 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:30.938063 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:31.066781 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:31.066944 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:31.196096 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:31.328009 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:31.328310 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:31.453941 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:31.454603 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:31.818350 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:31.917228 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:31.917393 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:32.066791 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:32.067071 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:32.242315 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:32.372575 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:32.372710 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:32.515137 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:32.515825 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:32.701303 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:32.810284 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:32.810375 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:33.012740 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:33.013134 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:33.175572 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:33.309639 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:33.309639 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:33.437974 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:33.438115 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:33.509684 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:33:33.612419 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:33:33.612947 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:33:33.765934 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:33:33.766096 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:33:33.771017 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:33:33.777106 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:33:33.777301 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:33:33.781135 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:33:33.781315 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:33:33.784628 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:145}} Nov 25 13:33:33.784960 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1393287685, sha256file filename: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:33:33.842368 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:33:33.842886 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1393287685, metadata filename: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:33:33.868782 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "456c2964-56d4-4356-8ad1-d93afa40def5",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1647097626",\n "created_at": "2025-11-25 18:33:16+00:00",\n "objects": [\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "82571f7121cb9475563edde615e4e4df",\n "offset": 0\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "a35891500449df17e893f84bd88472fb",\n "offset": 104857600\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "caad167b-7322-4d3c-918b-1ef8530687c1",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"caad167b-7322-4d3c-918b-1ef8530687c1\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"39069811c8de4688bdc78baf4cf5dc3b\\", \\"project_id\\": \\"8ad212ee97684fec9bf2f2dc25f47549\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:33:14.000000\\", \\"launched_at\\": \\"2025-11-25T18:33:16.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-376110385\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 0\\", \\"provider_auth\\": \\"CHAP Yx5aT3bCRPy9n2YRWwbp oNpP5h9jJZcvZXF4\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": true, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:33:14.000000\\", \\"updated_at\\": \\"2025-11-25T18:33:18.000000\\"}, \\"volume-glance-metadata\\": {\\"hw_rng_model\\": \\"virtio\\", \\"owner_specified.openstack.md5\\": \\"\\", \\"owner_specified.openstack.sha256\\": \\"\\", \\"owner_specified.openstack.object\\": \\"images/cirros-0.6.3-x86_64-disk\\", \\"os_glance_importing_to_stores\\": \\"\\", \\"os_glance_failed_import\\": \\"\\", \\"image_id\\": \\"95722938-8ac2-4a78-b8c9-2b052e3b4537\\", \\"image_name\\": \\"cirros-0.6.3-x86_64-disk\\", \\"checksum\\": \\"87617e24a5e30cb3b87fda8c0764838f\\", \\"container_format\\": \\"bare\\", \\"disk_format\\": \\"qcow2\\", \\"min_disk\\": \\"0\\", \\"min_ram\\": \\"0\\", \\"size\\": \\"21692416\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:33:33.871811 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] backup 456c2964-56d4-4356-8ad1-d93afa40def5 finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:33:33.938275 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:33.961866 np0005535648.novalocal sudo[149522]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 25 13:33:34.295661 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.357s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.297108 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1', 'target_lun': 0, 'volume_id': 'caad167b-7322-4d3c-918b-1ef8530687c1', 'auth_method': 'CHAP', 'auth_username': 'Yx5aT3bCRPy9n2YRWwbp', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405e8ca4c1b423c4243b8d75eec3', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:33:34.297413 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:33:34.298057 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:33:34.299574 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.307297 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.307499 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[453257e6-1568-4f2f-859f-d68d05368dc4]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:33:34.307922 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:34.308222 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:33:34.308222 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:33:34.308222 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:33:34.308457 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:33:34.309294 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.320240 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.320422 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a42e9d1-b86e-4917-b5b5-56ffcb38338b]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.321363 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.329728 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.329928 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6b96948-ed8f-4daf-a88e-ca9bfc2d6f48]: (4, ('tcp: [24] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.330279 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [24] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 (non-flash) Nov 25 13:33:34.330279 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:34.330460 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [24] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 (non-flash) Nov 25 13:33:34.330460 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:33:34.331168 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bdca03c40>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1'): ({'sda'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:33:34.331466 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sda {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:33:34.332267 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.337747 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e48366c0-ee65-4351-8bea-ab6469608471]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:33:34.339712 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:33:34.341447 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e48366c0-ee65-4351-8bea-ab6469608471]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.342151 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sda {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:33:34.343181 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.349245 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.006s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.349511 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8b77f65-0829-42b4-acd6-5cdd2d7ab924]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.350001 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:33:34.350988 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.406568 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.055s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.406867 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[79d1d0fc-7696-4afb-89ff-9754a656a6dd]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.407345 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:33:34.407797 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:33:34.408190 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:33:34.409405 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.420264 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.420502 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b533856-618b-40a8-baa5-a39f044743b9]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.421043 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:34.422011 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.455895 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --logout" returned: 0 in 0.033s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.456670 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[abc9363e-a404-4ffe-86d7-0c984e54e4e0]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1, portal: 38.102.83.58,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.457192 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1, portal: 38.102.83.58,3260] Nov 25 13:33:34.457192 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1, portal: 38.102.83.58,3260] successful. Nov 25 13:33:34.457192 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:34.459189 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:34.469911 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 -p 38.102.83.58:3260 --op delete" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:34.470106 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8423ff44-36ee-4cca-ac62-cb42877bbe62]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:34.470497 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:34.470841 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.173s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:33:34.471051 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (173ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:33:34.618649 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:33:34.619280 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:33:37.994933 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-c5c787e2-09c7-472e-b9bf-f9c4801de0b9 tempest-VolumesBackupsTest-210128247 None] Create backup finished. backup: 456c2964-56d4-4356-8ad1-d93afa40def5. Nov 25 13:33:39.590837 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Restore backup started, backup: 456c2964-56d4-4356-8ad1-d93afa40def5 volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. Nov 25 13:33:39.601982 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:33:39.602391 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:33:39.602841 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:33:39.604176 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:39.612731 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:39.613038 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[365f3174-a5bf-4183-8cd9-ddcf755c8dae]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[36ac38bb-fa63-4f46-8bd7-8dcdecbaf194]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:39.616159 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:33:39.616742 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[36ac38bb-fa63-4f46-8bd7-8dcdecbaf194]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:39.617194 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:33:39.617526 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:33:39.618288 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:39.627223 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:39.627417 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e3906b0-231a-45cd-b489-cd60b3e67cf8]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:39.628435 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:39.638606 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:39.639026 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4f84aee-c653-467f-9cfd-a0a1e4abc694]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:39.640536 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea3d4b51-1245-433c-bd4f-118b2751795f]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:39.641358 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:39.653721 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:33:39.654348 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:33:39.657211 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:33:39.658625 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40'] Nov 25 13:33:39.659087 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:33:39.659629 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:33:39.660495 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (57ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:33:43.674361 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:33:43.675600 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'target_lun': 0, 'volume_id': 'c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'auth_method': 'CHAP', 'auth_username': 'This3avNTH2434gUKoej', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:33:43.675902 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:33:43.676634 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:33:43.676907 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'target_lun': 0, 'volume_id': 'c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'auth_method': 'CHAP', 'auth_username': 'This3avNTH2434gUKoej', 'auth_password': '2vqJCWwLcyRoUno2', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:33:43.678162 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:43.683371 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[33004dc8-2efb-45a8-81f6-d621077eb86f]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:33:43.685341 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:33:43.686946 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[33004dc8-2efb-45a8-81f6-d621077eb86f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:43.687763 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:33:43.688530 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:33:43.688964 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:33:43.689855 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:43.697546 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260" returned: 21 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:43.697815 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffe40d48-6cf1-4b87-ae9d-9001deedae27]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:43.698299 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:33:43.698299 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:43.699218 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:43.712671 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:43.712953 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e7ad1bf-6eec-43c8-9724-10e91414359a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:43.713442 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e] added Nov 25 13:33:43.713442 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:33:43.713442 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:33:43.713442 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:33:43.713442 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:43.713776 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:33:43.714405 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.025(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:33:43.714782 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:33:44.716653 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:33:44.719168 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.731173 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.731508 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[43397bc2-e7dd-4f27-b15b-f93e9e7276fd]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:33:44.733527 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.736452 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.746755 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.747236 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8559d39b-5365-442e-a323-6ba3775fb517]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.747747 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.748782 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.761407 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.761752 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0602ffb-5430-4b5b-9525-a3f5ad2e5791]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.762153 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.763056 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.auth.username -v This3avNTH2434gUKoej {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.775592 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.auth.username -v This3avNTH2434gUKoej" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.775900 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f753d0fb-f2ff-46ff-913b-d41091630b98]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.776295 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'This3avNTH2434gUKoej'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.777340 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.789673 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.790032 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9bdeaaa-b874-4ca3-bb1c-ee42fa582d35]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.790584 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.791533 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.803012 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.803225 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1c023ea-b6a5-443c-8c78-22ecc3a2ad26]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.803708 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 25 13:33:44.803708 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:44.804051 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 25 13:33:44.804051 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:33:44.804311 np0005535648.novalocal cinder-backup[107475]: WARNING os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 25 13:33:44.804311 np0005535648.novalocal cinder-backup[107475]: Nov 25 13:33:44.805201 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.826097 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --login" returned: 0 in 0.019s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.826097 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2a3dba7-66a8-4935-9a7f-31bf80d69822]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.826097 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, portal: 38.102.83.58,3260] successful. Nov 25 13:33:44.826097 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.826976 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.840763 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.840976 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[32bac92b-fb69-400c-8b99-0a0a4653f84b]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.841429 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:44.842464 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.856865 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.857234 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2608300-3295-466b-a533-cd61aecddc89]: (4, ('tcp: [25] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.857927 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [25] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e (non-flash) Nov 25 13:33:44.857927 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:44.858357 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [25] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e (non-flash) Nov 25 13:33:44.858357 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:33:44.859120 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.171s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:33:44.859495 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:33:44.860601 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] HCTL ('2', '-', '-', 0) found on session 25 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:33:44.860982 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:33:44.862202 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:44.891629 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.027s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:44.891629 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[713f1dc2-558d-4215-90ad-c487e917ae6f]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:44.891629 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 25 and hctl ['2', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:33:45.895962 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 25 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:33:45.896914 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'target_lun': 0, 'volume_id': 'c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'auth_method': 'CHAP', 'auth_username': 'This3avNTH2434gUKoej', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:33:45.897474 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.221s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:33:45.899480 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2221ms) {'type': 'block', 'scsi_wwn': '36001405f3681ef689e9413081196c755', 'path': '/dev/sda'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:33:45.900370 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:45.922969 np0005535648.novalocal sudo[149715]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 25 13:33:46.262913 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.362s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:46.263311 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] starting restore of backup volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5 container: tempest-volumesbackupstest-backup-container-1393287685, to new volume c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, backup: 456c2964-56d4-4356-8ad1-d93afa40def5. {{(pid=107475) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:783}} Nov 25 13:33:46.263590 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1393287685, metadata filename: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_metadata. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] _read_metadata finished. Metadata: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "backup_description": null, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "backup_id": "456c2964-56d4-4356-8ad1-d93afa40def5", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "backup_name": "tempest-VolumesBackupsTest-Backup-1647097626", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "created_at": "2025-11-25 18:33:16+00:00", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "objects": [ Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "82571f7121cb9475563edde615e4e4df", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 0 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 52428800 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "a35891500449df17e893f84bd88472fb", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 104857600 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 157286400 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 209715200 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 262144000 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 314572800 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 367001600 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 419430400 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 471859200 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 524288000 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 576716800 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 629145600 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 681574400 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 734003200 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 786432000 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 838860800 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 891289600 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 943718400 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 996147200 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021": { Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "length": 25165824, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "offset": 1048576000 Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: ], Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "parent_id": null, Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "version": "1.0.0", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_id": "caad167b-7322-4d3c-918b-1ef8530687c1", Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"caad167b-7322-4d3c-918b-1ef8530687c1\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"39069811c8de4688bdc78baf4cf5dc3b\", \"project_id\": \"8ad212ee97684fec9bf2f2dc25f47549\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-25T18:33:14.000000\", \"launched_at\": \"2025-11-25T18:33:16.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-376110385\", \"display_description\": null, \"provider_location\": \"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 0\", \"provider_auth\": \"CHAP Yx5aT3bCRPy9n2YRWwbp oNpP5h9jJZcvZXF4\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"42ef2ac8-46e2-415f-ad47-5f278d4fe376\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": true, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"e7e65488-c10f-4744-a3b2-5f726bb311ba\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-25T18:33:14.000000\", \"updated_at\": \"2025-11-25T18:33:18.000000\"}, \"volume-glance-metadata\": {\"hw_rng_model\": \"virtio\", \"owner_specified.openstack.md5\": \"\", \"owner_specified.openstack.sha256\": \"\", \"owner_specified.openstack.object\": \"images/cirros-0.6.3-x86_64-disk\", \"os_glance_importing_to_stores\": \"\", \"os_glance_failed_import\": \"\", \"image_id\": \"95722938-8ac2-4a78-b8c9-2b052e3b4537\", \"image_name\": \"cirros-0.6.3-x86_64-disk\", \"checksum\": \"87617e24a5e30cb3b87fda8c0764838f\", \"container_format\": \"bare\", \"disk_format\": \"qcow2\", \"min_disk\": \"0\", \"min_ram\": \"0\", \"size\": \"21692416\"}}" Nov 25 13:33:46.301647 np0005535648.novalocal cinder-backup[107475]: }. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 25 13:33:46.306240 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Restoring backup version 1.0.0 {{(pid=107475) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:796}} Nov 25 13:33:46.306240 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1393287685, metadata filename: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_metadata. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] _read_metadata finished. Metadata: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "backup_description": null, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "backup_id": "456c2964-56d4-4356-8ad1-d93afa40def5", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "backup_name": "tempest-VolumesBackupsTest-Backup-1647097626", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "created_at": "2025-11-25 18:33:16+00:00", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "objects": [ Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "82571f7121cb9475563edde615e4e4df", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 0 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 52428800 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "a35891500449df17e893f84bd88472fb", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 104857600 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 157286400 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 209715200 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 262144000 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 314572800 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 367001600 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 419430400 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 471859200 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 524288000 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 576716800 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 629145600 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 681574400 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 734003200 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 786432000 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 838860800 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 891289600 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 943718400 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 996147200 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021": { Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "length": 25165824, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "offset": 1048576000 Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: ], Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "parent_id": null, Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "version": "1.0.0", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_id": "caad167b-7322-4d3c-918b-1ef8530687c1", Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"caad167b-7322-4d3c-918b-1ef8530687c1\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"39069811c8de4688bdc78baf4cf5dc3b\", \"project_id\": \"8ad212ee97684fec9bf2f2dc25f47549\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-25T18:33:14.000000\", \"launched_at\": \"2025-11-25T18:33:16.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-376110385\", \"display_description\": null, \"provider_location\": \"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 iqn.2010-10.org.openstack:volume-caad167b-7322-4d3c-918b-1ef8530687c1 0\", \"provider_auth\": \"CHAP Yx5aT3bCRPy9n2YRWwbp oNpP5h9jJZcvZXF4\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"42ef2ac8-46e2-415f-ad47-5f278d4fe376\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": true, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"e7e65488-c10f-4744-a3b2-5f726bb311ba\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-25T18:33:14.000000\", \"updated_at\": \"2025-11-25T18:33:18.000000\"}, \"volume-glance-metadata\": {\"hw_rng_model\": \"virtio\", \"owner_specified.openstack.md5\": \"\", \"owner_specified.openstack.sha256\": \"\", \"owner_specified.openstack.object\": \"images/cirros-0.6.3-x86_64-disk\", \"os_glance_importing_to_stores\": \"\", \"os_glance_failed_import\": \"\", \"image_id\": \"95722938-8ac2-4a78-b8c9-2b052e3b4537\", \"image_name\": \"cirros-0.6.3-x86_64-disk\", \"checksum\": \"87617e24a5e30cb3b87fda8c0764838f\", \"container_format\": \"bare\", \"disk_format\": \"qcow2\", \"min_disk\": \"0\", \"min_ram\": \"0\", \"size\": \"21692416\"}}" Nov 25 13:33:46.320689 np0005535648.novalocal cinder-backup[107475]: }. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 25 13:33:46.325343 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] v1 volume backup restore of 456c2964-56d4-4356-8ad1-d93afa40def5 started. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:700}} Nov 25 13:33:46.325343 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] metadata_object_names = ['volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021']. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:707}} Nov 25 13:33:46.385109 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_metadata', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:33:46.390197 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:33:46.394974 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:46.488718 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:46.880478 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:46.901990 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:47.107585 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:47.129405 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:47.331479 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:47.346348 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:47.561732 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:47.579819 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:47.758596 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:47.776662 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:47.984899 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:48.008886 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:48.187868 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:48.206077 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:48.402207 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:48.418508 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:48.622758 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:48.637831 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:48.823683 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:48.841201 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:49.024694 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:49.042819 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:49.231846 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:49.247041 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:49.426597 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:49.440872 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:49.626590 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:49.640620 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:49.819421 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:49.835852 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:50.042361 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:50.060833 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:50.242409 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:50.262254 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:50.462166 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:50.479298 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:50.684282 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:50.705376 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:50.925750 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restoring object. backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, object name: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021, volume: c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:33:50.939872 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:33:51.033019 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] v1 volume backup restore of 456c2964-56d4-4356-8ad1-d93afa40def5 finished. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:772}} Nov 25 13:33:51.033227 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Restoring volume base metadata {{(pid=107475) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 25 13:33:51.042590 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] No metadata of type 'volume-metadata' to restore {{(pid=107475) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 25 13:33:51.042990 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Restoring volume glance metadata {{(pid=107475) _restore_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:258}} Nov 25 13:33:51.106667 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] restore 456c2964-56d4-4356-8ad1-d93afa40def5 to c0b6b03b-5a95-42a3-9425-f24d0d90ef4e finished. {{(pid=107475) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:837}} Nov 25 13:33:51.117457 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.141700 np0005535648.novalocal sudo[149741]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 25 13:33:51.463934 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.346s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.465461 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'target_lun': 0, 'volume_id': 'c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 'auth_method': 'CHAP', 'auth_username': 'This3avNTH2434gUKoej', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405f3681ef689e9413081196c755', 'path': '/dev/sda'}), 'kwargs': {'force': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:33:51.465881 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:33:51.466931 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:33:51.468843 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.478009 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.478270 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f25e364-c711-4c6a-a67e-5ef7cb9b5780]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:33:51.478830 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:51.479110 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:33:51.479110 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:33:51.479110 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:33:51.479641 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:33:51.480606 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.488188 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.488465 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbfc189a-84a3-4a7c-9589-370726cbe071]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.489532 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.498318 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.498683 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[91e19ea7-e4d2-4fd2-b7b4-d4bc701aa972]: (4, ('tcp: [25] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.499175 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [25] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e (non-flash) Nov 25 13:33:51.499175 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:33:51.499442 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [25] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e (non-flash) Nov 25 13:33:51.499442 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:33:51.500484 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bdca4e340>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e'): ({'sda'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:33:51.500929 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sda {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:33:51.501930 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.507263 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[087c9cea-68a4-4bcf-812e-dae5095e5070]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:33:51.509235 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:33:51.510886 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[087c9cea-68a4-4bcf-812e-dae5095e5070]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.511624 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sda {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:33:51.512619 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.518756 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.006s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.519021 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cab16c63-b947-4e9d-ba34-40e9fd544a0b]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.519507 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:33:51.520597 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.586582 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.065s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.586951 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[307df566-7d67-4322-80d5-b30bf4206cdf]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.587529 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:33:51.587962 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:33:51.588482 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:33:51.589742 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.600273 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.600647 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec34f64e-0099-4676-b485-dc1ae7e402d4]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.601090 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:51.602155 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.628626 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --logout" returned: 0 in 0.026s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.628887 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f46e0e89-2756-43a9-8ed0-2e3c7bcc46a7]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, portal: 38.102.83.58,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.629192 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, portal: 38.102.83.58,3260] Nov 25 13:33:51.629192 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e, portal: 38.102.83.58,3260] successful. Nov 25 13:33:51.629192 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:51.630317 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:33:51.640847 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-c0b6b03b-5a95-42a3-9425-f24d0d90ef4e -p 38.102.83.58:3260 --op delete" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:33:51.641123 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[4495e055-f08d-4f57-8bde-62a932c417d8]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:33:51.641655 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:33:51.642337 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.176s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:33:51.642685 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (176ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:33:55.160706 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.manager [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Encryption key ID for volume c0b6b03b-5a95-42a3-9425-f24d0d90ef4e already matches encryption key ID in backup 456c2964-56d4-4356-8ad1-d93afa40def5. {{(pid=107475) _run_restore /opt/stack/cinder/cinder/backup/manager.py:874}} Nov 25 13:33:55.177507 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-35d6d979-ae0c-4620-b910-81c06806de6d tempest-VolumesBackupsTest-210128247 None] Finished restoring backup 456c2964-56d4-4356-8ad1-d93afa40def5 to volume c0b6b03b-5a95-42a3-9425-f24d0d90ef4e. Nov 25 13:33:58.384468 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] Delete backup started, backup: 456c2964-56d4-4356-8ad1-d93afa40def5. Nov 25 13:33:58.385967 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:33:58.386224 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:33:58.386511 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] delete started, backup: 456c2964-56d4-4356-8ad1-d93afa40def5, container: tempest-volumesbackupstest-backup-container-1393287685, prefix: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:33:58.427617 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_metadata', 'volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:33:58.459458 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00001 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.489812 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00002 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.521457 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00003 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.549646 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00004 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.572193 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00005 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.601824 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00006 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.657944 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00007 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.709845 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00008 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.739751 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00009 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.762812 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00010 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.787644 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00011 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.812220 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00012 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.836239 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00013 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.861147 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00014 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.892766 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00015 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.919336 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00016 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.944726 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00017 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.972469 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00018 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:58.997708 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00019 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:59.023055 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00020 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:59.047811 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5-00021 in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:59.075918 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_metadata in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:59.106960 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] deleted object: volume_caad167b-7322-4d3c-918b-1ef8530687c1/20251125183323/az_nova_backup_456c2964-56d4-4356-8ad1-d93afa40def5_sha256file in container: tempest-volumesbackupstest-backup-container-1393287685. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:33:59.107310 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] delete 456c2964-56d4-4356-8ad1-d93afa40def5 finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:33:59.110298 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:33:59.125575 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] Created reservations ['6b291e1c-4a15-4be3-aea6-c918f9043412', '14e0fe12-f281-4a84-a980-9be9fbef65e4'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:33:59.141136 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-28657a98-bb00-4924-83de-2fed2bfac0ef tempest-VolumesBackupsTest-210128247 None] Delete backup finished, backup 456c2964-56d4-4356-8ad1-d93afa40def5 deleted. Nov 25 13:34:02.980704 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Create backup started, backup: ba82f314-ba56-40ab-bb91-714edc54e969 volume: 784c424e-dbda-4cd1-adc5-738e89268c51. Nov 25 13:34:02.987042 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1505184479',created_at=2025-11-25T18:34:02Z,data_timestamp=2025-11-25T18:34:02Z,deleted=False,deleted_at=None,display_description='tempest-backup-description-72838921',display_name='tempest-VolumesBackupsTest-Backup-1430048120',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=ba82f314-ba56-40ab-bb91-714edc54e969,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='8ad212ee97684fec9bf2f2dc25f47549',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='39069811c8de4688bdc78baf4cf5dc3b',volume_id=784c424e-dbda-4cd1-adc5-738e89268c51) Nov 25 13:34:03.007638 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:34:03.007810 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:34:03.008061 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:34:03.009257 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:03.018905 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:03.019140 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[901adb6b-1281-4bf0-9e4d-5566b50754a5]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[1e810267-1aaf-4d2c-98ac-ffd17723d83d]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:03.021370 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:34:03.021822 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e810267-1aaf-4d2c-98ac-ffd17723d83d]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:03.022340 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:34:03.022610 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:34:03.023147 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:03.033217 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:03.033357 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ce827c5-bbe5-474e-b288-57422b0de094]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:03.034239 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:03.042623 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:03.042820 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[71530e9a-3165-44f4-a8bd-042e5967d2e0]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:03.043691 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7db149ba-32d6-4768-a4df-373784877f76]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:03.044118 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:03.054786 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:34:03.055057 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:34:03.056922 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:34:03.057475 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40'] Nov 25 13:34:03.057717 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:34:03.057936 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:34:03.058303 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (49ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:34:06.567227 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:34:06.568095 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51', 'target_lun': 0, 'volume_id': '784c424e-dbda-4cd1-adc5-738e89268c51', 'auth_method': 'CHAP', 'auth_username': 'ipkLnT9EFegPa9N62u2P', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:34:06.568413 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:34:06.569379 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:34:06.569726 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51', 'target_lun': 0, 'volume_id': '784c424e-dbda-4cd1-adc5-738e89268c51', 'auth_method': 'CHAP', 'auth_username': 'ipkLnT9EFegPa9N62u2P', 'auth_password': 'hQq2T6Jw5gu78Eqp', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:34:06.570799 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:06.578276 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d497c8c3-6b56-453e-b2f6-3c701f0ee456]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:34:06.579446 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:34:06.580189 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d497c8c3-6b56-453e-b2f6-3c701f0ee456]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:06.580988 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:34:06.581745 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:34:06.582118 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:34:06.583066 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:06.593739 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260" returned: 21 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:06.594006 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[787c6375-b81a-4396-80ab-6524dab2f121]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:06.594526 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:34:06.594526 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:06.595426 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:06.602958 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:06.603231 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd337064-ed72-4017-9bf5-2812846a12e1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:06.603757 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51] added Nov 25 13:34:06.603757 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:34:06.603757 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:34:06.603757 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:34:06.603757 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:06.604072 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:34:06.604734 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.022(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:34:06.605221 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:34:07.607384 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:34:07.609084 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.620903 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.621204 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebd0adda-f621-4105-9b5a-5530f50c8a96]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:34:07.622952 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.624167 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.636127 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.636270 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[915e5839-49b9-474e-8019-d2dc2cf38bd5]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.636814 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.637971 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.649388 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.649657 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc35eea2-ea45-479b-aaf5-75ea6224d074]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.650083 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.650977 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v ipkLnT9EFegPa9N62u2P {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.661998 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v ipkLnT9EFegPa9N62u2P" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.662248 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d298b787-4be2-4587-9c85-67293e08c0d1]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.662720 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ipkLnT9EFegPa9N62u2P'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.663775 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.675863 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.676113 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[96f13065-0ca3-4df4-a1ee-804aa90d6e1d]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.676789 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.677628 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.691517 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.691788 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[041c281f-e3d8-4caa-ad8f-255ad4f9c5c2]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.692213 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 25 13:34:07.692213 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:07.692509 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 25 13:34:07.692509 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:34:07.692837 np0005535648.novalocal cinder-backup[107475]: WARNING os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 25 13:34:07.692837 np0005535648.novalocal cinder-backup[107475]: Nov 25 13:34:07.693709 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.711944 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --login" returned: 0 in 0.017s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.712335 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[40321a7a-037c-4d47-955b-e8455787cc9d]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.713029 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51, portal: 38.102.83.58,3260] successful. Nov 25 13:34:07.713029 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.714326 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.725896 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.726173 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c0bf1ba-5095-4b19-bbd5-045c4cb8abd9]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.726799 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:07.727819 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.739246 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.739495 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdee72d5-a538-4043-bd3c-51355c9a9c83]: (4, ('tcp: [26] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.740087 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [26] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 (non-flash) Nov 25 13:34:07.740087 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:07.740414 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [26] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 (non-flash) Nov 25 13:34:07.740414 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:34:07.741022 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.159s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:34:07.741405 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:34:07.742185 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] HCTL ('2', '-', '-', 0) found on session 26 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:34:07.742494 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:34:07.743477 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:07.775076 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.031s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:07.775415 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6d9a757-04c5-4a27-85c0-bd7b786ee9ee]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:07.776132 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 26 and hctl ['2', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:34:08.778468 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 26 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:34:08.779104 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51', 'target_lun': 0, 'volume_id': '784c424e-dbda-4cd1-adc5-738e89268c51', 'auth_method': 'CHAP', 'auth_username': 'ipkLnT9EFegPa9N62u2P', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:34:08.779534 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.211s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:34:08.779732 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2211ms) {'type': 'block', 'scsi_wwn': '36001405193fd7c6a31a401e8849d8beb', 'path': '/dev/sda'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:34:08.780106 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:08.806369 np0005535648.novalocal sudo[150021]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 25 13:34:09.096933 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.316s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:09.103863 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:34:09.106915 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=98,cinder:UPDATE=18,cinder:INSERT=17 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:34:09.117379 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1505184479,backup: ba82f314-ba56-40ab-bb91-714edc54e969. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:34:09.188346 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] generate_object_name_prefix: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969 {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:34:09.193701 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] starting backup of volume: 784c424e-dbda-4cd1-adc5-738e89268c51, volume size: 1073741824, object names prefix volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:34:09.605856 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:09.751100 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:09.751351 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:09.907876 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:09.908383 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:10.132174 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:10.271663 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:10.271961 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:10.426832 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:10.426991 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:10.820666 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:10.953847 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:10.954339 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:11.133932 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:11.134194 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:11.532919 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:11.670846 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:11.671089 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:11.826116 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:11.826395 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:12.065447 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:12.198311 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:12.198599 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:12.351786 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:12.352093 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:12.936272 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:13.047384 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:13.047663 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:13.185036 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:13.185389 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:13.557133 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:13.694737 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:13.694893 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:13.826446 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:13.826599 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:13.946924 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:14.043889 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:14.044079 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:14.148029 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:14.148158 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:14.488322 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:14.621968 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:14.622353 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:14.768320 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:14.768320 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:15.347778 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:15.447436 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:15.447611 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:15.569242 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:15.569671 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:15.734733 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:15.860027 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:15.860027 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:16.008988 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:16.009119 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:16.177485 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:16.286980 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:16.287123 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:16.442842 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:16.443009 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:16.561018 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:16.704973 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:16.705611 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:16.848877 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:16.849145 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:17.002071 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:17.100595 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:17.101415 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:17.229287 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:17.229621 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:17.402597 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:17.520835 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:17.521056 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:17.679712 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:17.679923 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:18.304758 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:18.403556 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:18.403675 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:18.556473 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:18.556697 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:19.155740 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:19.353596 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:19.354294 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:19.509799 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:19.510037 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:19.910058 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:20.043708 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:20.043978 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:20.180129 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:20.180392 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:20.567512 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:20.689079 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:20.689168 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:20.795201 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:20.795510 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:21.211378 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:21.357989 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:21.357989 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:21.505044 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:21.505317 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:21.595969 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:34:21.659891 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:34:21.660037 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:34:21.737235 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:34:21.737373 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:34:21.742875 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:34:21.748910 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:34:21.749126 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:34:21.753419 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:121}} Nov 25 13:34:21.753694 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:34:21.757067 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:34:21.757406 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1505184479, sha256file filename: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:34:21.846302 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:34:21.847181 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1505184479, metadata filename: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:34:21.886304 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] _write_metadata finished. Metadata: b'{\n "backup_description": "tempest-backup-description-72838921",\n "backup_id": "ba82f314-ba56-40ab-bb91-714edc54e969",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1430048120",\n "created_at": "2025-11-25 18:34:02+00:00",\n "objects": [\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "784c424e-dbda-4cd1-adc5-738e89268c51",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"784c424e-dbda-4cd1-adc5-738e89268c51\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"39069811c8de4688bdc78baf4cf5dc3b\\", \\"project_id\\": \\"8ad212ee97684fec9bf2f2dc25f47549\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:34:01.000000\\", \\"launched_at\\": \\"2025-11-25T18:34:02.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1244734299\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 0\\", \\"provider_auth\\": \\"CHAP ipkLnT9EFegPa9N62u2P hQq2T6Jw5gu78Eqp\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:34:01.000000\\", \\"updated_at\\": \\"2025-11-25T18:34:05.000000\\"}, \\"volume-metadata\\": {\\"vol-meta1\\": \\"value1\\", \\"vol-meta2\\": \\"value2\\", \\"vol-meta3\\": \\"value3\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:34:21.891087 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] backup ba82f314-ba56-40ab-bb91-714edc54e969 finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:34:21.999226 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.029304 np0005535648.novalocal sudo[150041]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 25 13:34:22.394099 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.395s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.395230 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51', 'target_lun': 0, 'volume_id': '784c424e-dbda-4cd1-adc5-738e89268c51', 'auth_method': 'CHAP', 'auth_username': 'ipkLnT9EFegPa9N62u2P', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405193fd7c6a31a401e8849d8beb', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:34:22.395502 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:34:22.396003 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:34:22.397256 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.409553 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.409750 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ad2bd37-32fc-409e-901d-3fa0f4ab5cef]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:34:22.410124 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:22.410445 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:34:22.410445 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:34:22.410445 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:34:22.410705 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:34:22.411425 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.420794 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.421026 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d04d48d-d995-4d1f-a94d-551c6cb8979b]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.421777 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.434937 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.435144 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[5744a76d-3296-42a5-8d60-3edbbfb4c951]: (4, ('tcp: [26] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.435508 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [26] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 (non-flash) Nov 25 13:34:22.435508 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:22.435727 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [26] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 (non-flash) Nov 25 13:34:22.435727 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:34:22.436441 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bd9a6a160>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51'): ({'sda'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:34:22.436730 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sda {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:34:22.437385 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.445037 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[e8bbd5e1-8d17-4099-aa4a-f4638ec39e76]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:34:22.445803 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:34:22.447193 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8bbd5e1-8d17-4099-aa4a-f4638ec39e76]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.448412 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sda {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:34:22.449724 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.458800 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.459370 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[51473525-48f6-4f83-8805-52d8f09ded1e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.460184 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:34:22.461876 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.521244 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.059s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.521531 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[49474e7d-a890-4a6f-9473-6d5a3793398e]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.522016 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:34:22.522489 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:34:22.522967 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:34:22.524168 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.538354 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.538558 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[65ef3592-954f-487c-8f56-50cf17008ff7]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.539055 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:22.539964 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.572223 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --logout" returned: 0 in 0.032s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.572643 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c5df0c9-9247-4098-803d-c884083c0362]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51, portal: 38.102.83.58,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.572955 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51, portal: 38.102.83.58,3260] Nov 25 13:34:22.572955 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51, portal: 38.102.83.58,3260] successful. Nov 25 13:34:22.572955 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:22.573748 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:22.582762 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 -p 38.102.83.58:3260 --op delete" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:22.583052 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[65abb90a-97f9-476d-832e-6e737d309a51]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:22.583312 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:22.583619 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.188s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:34:22.583805 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (188ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:34:26.108336 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-26353c49-8e0e-4eab-8362-0055ed433ba7 tempest-VolumesBackupsTest-210128247 None] Create backup finished. backup: ba82f314-ba56-40ab-bb91-714edc54e969. Nov 25 13:34:27.969263 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Restore backup started, backup: ba82f314-ba56-40ab-bb91-714edc54e969 volume: 298f53d0-7817-4d61-91ca-c79899e9b937. Nov 25 13:34:27.981513 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:34:27.981778 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:34:27.982327 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:34:27.984853 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:27.993295 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:27.993566 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8641daa8-a87c-4dd7-8520-63acfbcb71e7]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[577f4824-b55a-41f1-ad46-dcd16c93a825]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:27.996271 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:34:27.997384 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[577f4824-b55a-41f1-ad46-dcd16c93a825]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:27.998394 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:34:27.999260 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:34:28.000602 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:28.011718 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:28.011941 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b649186-8e53-4fb4-b119-fb1ebe9f423c]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:28.013271 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:28.024597 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:28.024802 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6665715-c7e0-4a64-90f1-c70171de5b51]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:28.025999 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a39df3a6-202b-41b7-b307-a523717d62c9]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:28.026683 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:28.038980 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:34:28.039351 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:34:28.041354 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:34:28.042353 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40'] Nov 25 13:34:28.042687 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:34:28.042982 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:34:28.043571 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (60ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:34:32.059028 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:34:32.060166 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937', 'target_lun': 0, 'volume_id': '298f53d0-7817-4d61-91ca-c79899e9b937', 'auth_method': 'CHAP', 'auth_username': 'd5y3v72ZtQm79oVTnGqH', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:34:32.060467 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:34:32.061116 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:34:32.061403 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937', 'target_lun': 0, 'volume_id': '298f53d0-7817-4d61-91ca-c79899e9b937', 'auth_method': 'CHAP', 'auth_username': 'd5y3v72ZtQm79oVTnGqH', 'auth_password': '4pzrPoTquH6HCyTv', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:34:32.062487 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:32.068964 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f402dab1-cdaa-4cdd-84dc-4246d755e274]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:34:32.070458 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:34:32.071366 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f402dab1-cdaa-4cdd-84dc-4246d755e274]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:32.072213 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:34:32.072851 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:34:32.073236 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:34:32.074199 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:32.083811 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260" returned: 21 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:32.084083 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b212c9e9-e247-4311-94ee-b2998835df74]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:32.084513 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:34:32.084513 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:32.085448 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:32.096353 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:32.096711 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3711e1f-fb52-46f0-b91c-d13abef8c4ae]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:32.097236 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937] added Nov 25 13:34:32.097236 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:34:32.097236 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:34:32.097236 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:34:32.097236 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:32.097504 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:34:32.098143 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.025(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:34:32.098515 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:34:33.100209 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:34:33.101728 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.111482 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.111807 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[bab5a72f-aaa9-4aa5-899e-6b50df10ac0c]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:34:33.113124 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.115886 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.124695 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.124987 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[35392533-5932-493c-a257-d5c44f4e50dd]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.125493 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.126449 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.134402 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.134743 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3896c52e-6b8f-4eef-8e32-f74993fca9f4]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.135248 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.136177 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v d5y3v72ZtQm79oVTnGqH {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.143796 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v d5y3v72ZtQm79oVTnGqH" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.144083 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2dfbaf50-b777-4f6b-9b3a-382a51f0ff46]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.144612 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'd5y3v72ZtQm79oVTnGqH'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.145591 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.153346 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.153696 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7000565d-0a18-4e42-b804-59dd2202f8b9]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.154325 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.155279 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.163696 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.163991 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c450430-6ff1-4800-9b96-a678a226dc00]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.164462 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 25 13:34:33.164462 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:33.164758 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 25 13:34:33.164758 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:34:33.165052 np0005535648.novalocal cinder-backup[107475]: WARNING os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 25 13:34:33.165052 np0005535648.novalocal cinder-backup[107475]: Nov 25 13:34:33.166041 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.180414 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --login" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.181021 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[196851c3-1bd0-44be-9fa7-ea82f8aecb45]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.181482 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937, portal: 38.102.83.58,3260] successful. Nov 25 13:34:33.181482 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.182466 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.192764 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.192911 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a22126e-a137-49ee-8c07-94c6df6be6eb]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.193208 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:33.194026 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.206649 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.206921 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea2ea52f-688c-4dca-9766-579d2673c2e2]: (4, ('tcp: [27] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.207405 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [27] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 (non-flash) Nov 25 13:34:33.207405 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:33.207712 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [27] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 (non-flash) Nov 25 13:34:33.207712 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:34:33.208199 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.135s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:34:33.208468 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:34:33.209192 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] HCTL ('2', '-', '-', 0) found on session 27 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:34:33.209454 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:34:33.210564 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:33.238147 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.027s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:33.238147 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a872a28f-5845-44b9-a0ef-db6f20234796]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:33.238660 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 27 and hctl ['2', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:34:34.240467 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 27 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:34:34.241407 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937', 'target_lun': 0, 'volume_id': '298f53d0-7817-4d61-91ca-c79899e9b937', 'auth_method': 'CHAP', 'auth_username': 'd5y3v72ZtQm79oVTnGqH', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:34:34.242503 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.181s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:34:34.242923 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2182ms) {'type': 'block', 'scsi_wwn': '36001405a71ceebbbe8c40cea6146c464', 'path': '/dev/sda'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:34:34.243580 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:34.266252 np0005535648.novalocal sudo[150231]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 25 13:34:34.591837 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.347s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:34.592840 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] starting restore of backup volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969 container: tempest-volumesbackupstest-backup-container-1505184479, to new volume 298f53d0-7817-4d61-91ca-c79899e9b937, backup: ba82f314-ba56-40ab-bb91-714edc54e969. {{(pid=107475) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:783}} Nov 25 13:34:34.593199 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1505184479, metadata filename: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_metadata. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] _read_metadata finished. Metadata: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "backup_description": "tempest-backup-description-72838921", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "backup_id": "ba82f314-ba56-40ab-bb91-714edc54e969", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "backup_name": "tempest-VolumesBackupsTest-Backup-1430048120", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "created_at": "2025-11-25 18:34:02+00:00", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "objects": [ Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 0 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 52428800 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 104857600 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 157286400 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 209715200 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 262144000 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 314572800 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 367001600 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 419430400 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 471859200 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 524288000 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 576716800 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 629145600 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 681574400 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 734003200 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 786432000 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 838860800 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 891289600 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 943718400 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 996147200 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021": { Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "length": 25165824, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "offset": 1048576000 Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: ], Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "parent_id": null, Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "version": "1.0.0", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_id": "784c424e-dbda-4cd1-adc5-738e89268c51", Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"784c424e-dbda-4cd1-adc5-738e89268c51\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"39069811c8de4688bdc78baf4cf5dc3b\", \"project_id\": \"8ad212ee97684fec9bf2f2dc25f47549\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-25T18:34:01.000000\", \"launched_at\": \"2025-11-25T18:34:02.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-1244734299\", \"display_description\": null, \"provider_location\": \"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 0\", \"provider_auth\": \"CHAP ipkLnT9EFegPa9N62u2P hQq2T6Jw5gu78Eqp\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"42ef2ac8-46e2-415f-ad47-5f278d4fe376\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": false, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"e7e65488-c10f-4744-a3b2-5f726bb311ba\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-25T18:34:01.000000\", \"updated_at\": \"2025-11-25T18:34:05.000000\"}, \"volume-metadata\": {\"vol-meta1\": \"value1\", \"vol-meta2\": \"value2\", \"vol-meta3\": \"value3\"}}" Nov 25 13:34:34.621006 np0005535648.novalocal cinder-backup[107475]: }. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 25 13:34:34.625272 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Restoring backup version 1.0.0 {{(pid=107475) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:796}} Nov 25 13:34:34.625272 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] _read_metadata started, container name: tempest-volumesbackupstest-backup-container-1505184479, metadata filename: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_metadata. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:323}} Nov 25 13:34:34.625272 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:34:34.625272 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] _read_metadata finished. Metadata: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "backup_description": "tempest-backup-description-72838921", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "backup_id": "ba82f314-ba56-40ab-bb91-714edc54e969", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "backup_name": "tempest-VolumesBackupsTest-Backup-1430048120", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "created_at": "2025-11-25 18:34:02+00:00", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "objects": [ Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 0 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 52428800 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 104857600 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 157286400 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 209715200 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 262144000 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 314572800 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 367001600 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 419430400 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 471859200 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 524288000 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 576716800 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 629145600 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 681574400 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 734003200 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 786432000 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 838860800 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 891289600 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 943718400 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 52428800, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "25e317773f308e446cc84c503a6d1f85", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 996147200 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021": { Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "compression": "zlib", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "length": 25165824, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "md5": "77377273b0a4b61febdbf7bbf52b9db9", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "offset": 1048576000 Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: } Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: ], Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "parent_id": null, Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "version": "1.0.0", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_id": "784c424e-dbda-4cd1-adc5-738e89268c51", Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: "volume_meta": "{\"version\": 2, \"volume-base-metadata\": {\"id\": \"784c424e-dbda-4cd1-adc5-738e89268c51\", \"_name_id\": null, \"use_quota\": true, \"ec2_id\": null, \"user_id\": \"39069811c8de4688bdc78baf4cf5dc3b\", \"project_id\": \"8ad212ee97684fec9bf2f2dc25f47549\", \"snapshot_id\": null, \"cluster_name\": null, \"host\": \"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\", \"size\": 1, \"availability_zone\": \"nova\", \"status\": \"backing-up\", \"attach_status\": \"detached\", \"migration_status\": null, \"scheduled_at\": \"2025-11-25T18:34:01.000000\", \"launched_at\": \"2025-11-25T18:34:02.000000\", \"terminated_at\": null, \"display_name\": \"tempest-VolumesBackupsTest-Volume-1244734299\", \"display_description\": null, \"provider_location\": \"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 iqn.2010-10.org.openstack:volume-784c424e-dbda-4cd1-adc5-738e89268c51 0\", \"provider_auth\": \"CHAP ipkLnT9EFegPa9N62u2P hQq2T6Jw5gu78Eqp\", \"provider_geometry\": null, \"provider_id\": null, \"volume_type_id\": \"42ef2ac8-46e2-415f-ad47-5f278d4fe376\", \"source_volid\": null, \"encryption_key_id\": null, \"consistencygroup_id\": null, \"group_id\": null, \"bootable\": false, \"multiattach\": false, \"replication_status\": null, \"replication_extended_status\": null, \"replication_driver_data\": null, \"previous_status\": \"available\", \"service_uuid\": \"e7e65488-c10f-4744-a3b2-5f726bb311ba\", \"shared_targets\": false, \"deleted_at\": null, \"deleted\": false, \"created_at\": \"2025-11-25T18:34:01.000000\", \"updated_at\": \"2025-11-25T18:34:05.000000\"}, \"volume-metadata\": {\"vol-meta1\": \"value1\", \"vol-meta2\": \"value2\", \"vol-meta3\": \"value3\"}}" Nov 25 13:34:34.649066 np0005535648.novalocal cinder-backup[107475]: }. {{(pid=107475) _read_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:330}} Nov 25 13:34:34.651055 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] v1 volume backup restore of ba82f314-ba56-40ab-bb91-714edc54e969 started. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:700}} Nov 25 13:34:34.651055 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] metadata_object_names = ['volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021']. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:707}} Nov 25 13:34:34.718351 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_metadata', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:34:34.722970 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:34:34.727915 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:34.749950 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:34.993291 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:35.011477 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:35.234395 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:35.252682 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:35.449814 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:35.474298 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:35.715512 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:35.733400 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:35.981381 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:35.999019 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:36.192421 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:36.212480 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:36.409949 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:36.425722 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:36.612150 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:36.629957 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:36.871265 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:36.889905 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:37.121748 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:37.141780 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:37.332792 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:37.352301 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:37.530078 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:37.547197 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:37.726459 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:37.741610 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:37.934482 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:37.947523 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:38.141075 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:38.154232 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:38.345128 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:38.362786 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:38.564933 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:38.583276 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:38.788825 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:38.806583 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:38.984583 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:39.001189 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:39.203092 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restoring object. backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, object name: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021, volume: 298f53d0-7817-4d61-91ca-c79899e9b937. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:727}} Nov 25 13:34:39.221029 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] decompressing data using zlib algorithm {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:744}} Nov 25 13:34:39.312577 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] v1 volume backup restore of ba82f314-ba56-40ab-bb91-714edc54e969 finished. {{(pid=107475) _restore_v1 /opt/stack/cinder/cinder/backup/chunkeddriver.py:772}} Nov 25 13:34:39.312577 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Restoring volume base metadata {{(pid=107475) _restore_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:177}} Nov 25 13:34:39.319013 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Restoring volume metadata {{(pid=107475) _restore_vol_meta /opt/stack/cinder/cinder/backup/driver.py:249}} Nov 25 13:34:39.323894 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] No metadata of type 'volume-glance-metadata' to restore {{(pid=107475) put /opt/stack/cinder/cinder/backup/driver.py:345}} Nov 25 13:34:39.324035 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] restore ba82f314-ba56-40ab-bb91-714edc54e969 to 298f53d0-7817-4d61-91ca-c79899e9b937 finished. {{(pid=107475) restore /opt/stack/cinder/cinder/backup/chunkeddriver.py:837}} Nov 25 13:34:39.324370 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.351093 np0005535648.novalocal sudo[150237]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 25 13:34:39.676621 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.352s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.678604 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937', 'target_lun': 0, 'volume_id': '298f53d0-7817-4d61-91ca-c79899e9b937', 'auth_method': 'CHAP', 'auth_username': 'd5y3v72ZtQm79oVTnGqH', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '36001405a71ceebbbe8c40cea6146c464', 'path': '/dev/sda'}), 'kwargs': {'force': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:34:39.679008 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:34:39.679915 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:34:39.681751 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.692350 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.692659 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3d81f72-9642-43ed-b44f-62086a695e04]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:34:39.693239 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:39.693528 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:34:39.693528 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:34:39.693528 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:34:39.694028 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:34:39.694900 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.701831 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.702083 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c4a6c2a-1718-4cd7-b5fd-185b723b0aa9]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.703094 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.710863 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.711188 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[272c39aa-ca0e-4916-994e-b76218429d22]: (4, ('tcp: [27] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.711720 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [27] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 (non-flash) Nov 25 13:34:39.711720 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:34:39.712010 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [27] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 (non-flash) Nov 25 13:34:39.712010 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:34:39.712968 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bd9a42a20>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937'): ({'sda'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:34:39.713400 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sda {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:34:39.714278 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.720222 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ac3b9af6-3365-4800-a3e2-eacc555c00a3]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:34:39.722398 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:34:39.724134 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac3b9af6-3365-4800-a3e2-eacc555c00a3]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.724870 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sda {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:34:39.725840 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.732294 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.006s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.732688 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[68d279b7-406d-4652-bbf4-de602f0af6e5]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.733161 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:34:39.734027 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.784462 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.050s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.784876 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[227f489b-01cb-40ab-acfe-8c83ad64861f]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.785900 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:34:39.786307 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:34:39.786827 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:34:39.788269 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.801285 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.801525 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3fbdda5b-0393-4ced-88fb-81492ab14fe1]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.802213 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:39.803311 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.839674 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --logout" returned: 0 in 0.036s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.840000 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae233ad5-cc29-462b-a625-10cf3b28c139]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937, portal: 38.102.83.58,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.840608 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937, portal: 38.102.83.58,3260] Nov 25 13:34:39.840608 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937, portal: 38.102.83.58,3260] successful. Nov 25 13:34:39.840608 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:39.841760 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:34:39.853031 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-298f53d0-7817-4d61-91ca-c79899e9b937 -p 38.102.83.58:3260 --op delete" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:34:39.853349 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cf55065-1ead-49fe-bdfe-858deec4913e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:34:39.853814 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:34:39.854244 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.174s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:34:39.854568 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (175ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:34:43.376003 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.manager [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Encryption key ID for volume 298f53d0-7817-4d61-91ca-c79899e9b937 already matches encryption key ID in backup ba82f314-ba56-40ab-bb91-714edc54e969. {{(pid=107475) _run_restore /opt/stack/cinder/cinder/backup/manager.py:874}} Nov 25 13:34:43.401421 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-19c3ac89-ae34-46ff-ade1-d8313d32d7b4 tempest-VolumesBackupsTest-210128247 None] Finished restoring backup ba82f314-ba56-40ab-bb91-714edc54e969 to volume 298f53d0-7817-4d61-91ca-c79899e9b937. Nov 25 13:34:45.771388 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] Delete backup started, backup: ba82f314-ba56-40ab-bb91-714edc54e969. Nov 25 13:34:45.772453 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:34:45.773135 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:34:45.773361 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] delete started, backup: ba82f314-ba56-40ab-bb91-714edc54e969, container: tempest-volumesbackupstest-backup-container-1505184479, prefix: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:34:45.831587 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_metadata', 'volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:34:45.865920 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00001 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:45.893359 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00002 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:45.917324 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00003 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:45.943612 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00004 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:45.975008 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00005 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.008471 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00006 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.032224 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00007 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.059584 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00008 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.083906 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00009 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.113382 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00010 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.142578 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00011 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.173992 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00012 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.200647 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00013 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.227462 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00014 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.268466 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00015 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.298941 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00016 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.320241 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00017 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.344387 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00018 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.363719 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00019 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.382245 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00020 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.400494 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969-00021 in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.419731 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_metadata in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.448995 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_784c424e-dbda-4cd1-adc5-738e89268c51/20251125183409/az_nova_backup_ba82f314-ba56-40ab-bb91-714edc54e969_sha256file in container: tempest-volumesbackupstest-backup-container-1505184479. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:34:46.449173 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] delete ba82f314-ba56-40ab-bb91-714edc54e969 finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:34:46.451821 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:34:46.463892 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] Created reservations ['870fcc79-fe94-4f6f-b9f1-6da16cd452e5', 'e01ba2c4-8759-4835-9bc5-c4d7dbed3bd8'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:34:46.477072 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-71d90ed5-d5a6-49d1-8cb7-4343e9692ab5 tempest-VolumesBackupsTest-210128247 None] Delete backup finished, backup ba82f314-ba56-40ab-bb91-714edc54e969 deleted. Nov 25 13:34:56.477168 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:UPDATE=14,cinder:SELECT=65,cinder:INSERT=4 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:35:25.504828 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:35:34.621891 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:35:34.622346 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:35:42.550731 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Create backup started, backup: 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d volume: 6fd4cbf1-dd63-4003-908c-0042894cc452. Nov 25 13:35:42.572829 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-1326658444',created_at=2025-11-25T18:35:42Z,data_timestamp=2025-11-25T18:35:42Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-2042152272',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=0afbd6a6-8e37-4526-a7c6-18ac3231ae7d,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='8ad212ee97684fec9bf2f2dc25f47549',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='39069811c8de4688bdc78baf4cf5dc3b',volume_id=6fd4cbf1-dd63-4003-908c-0042894cc452) Nov 25 13:35:43.997574 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:35:43.998036 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:35:43.998425 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:35:43.999743 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:44.006895 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:44.007230 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7806a302-629e-40a6-b6ea-cdb56a98fec7]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[6c4b0c0b-9449-4a9b-9cab-fa788e4e7370]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:44.009868 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:35:44.010456 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c4b0c0b-9449-4a9b-9cab-fa788e4e7370]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:44.011085 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:35:44.011530 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:35:44.012370 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:44.022348 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:44.022609 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[84d59e1a-e03a-48f6-9d2e-192b7714b374]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:44.023897 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:44.034774 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:44.035052 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd746904-900d-46d3-a8aa-a9e6f3422437]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:44.036307 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[df0e7e26-dfa3-4808-ba49-4781251c02fe]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:44.036907 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:44.047808 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:35:44.048196 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:35:44.050022 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:35:44.051058 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fef2:8c0b', 'fe80::bc21:6dff:fe62:75c9'] Nov 25 13:35:44.051356 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:35:44.051658 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:35:44.052182 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (53ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fef2:8c0b', 'fe80::bc21:6dff:fe62:75c9']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:35:47.561622 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:35:47.562352 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93', 'target_lun': 0, 'volume_id': '4db5b17c-8deb-40e5-97e5-5e220df30a93', 'auth_method': 'CHAP', 'auth_username': 'WxNiQ5U4NP3SPA2nmzjf', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:35:47.562594 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:35:47.562965 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.000s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:35:47.563166 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93', 'target_lun': 0, 'volume_id': '4db5b17c-8deb-40e5-97e5-5e220df30a93', 'auth_method': 'CHAP', 'auth_username': 'WxNiQ5U4NP3SPA2nmzjf', 'auth_password': 'QyA8STkhDG3A5WZ9', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:35:47.564502 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:47.574706 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[146df15a-7b76-4b98-b9dc-67c4ed22cbf8]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:35:47.577210 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:35:47.579248 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[146df15a-7b76-4b98-b9dc-67c4ed22cbf8]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:47.580037 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:35:47.580750 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:35:47.581108 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:35:47.582139 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:47.593453 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260" returned: 21 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:47.593693 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f21db3b-6136-4329-bd54-948e41b23657]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:47.594570 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:35:47.594570 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:47.595882 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:47.605236 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:47.605451 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0461991-e35b-487b-82a0-28de6d9bf4f8]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:47.605885 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93] added Nov 25 13:35:47.605885 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:35:47.605885 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:35:47.605885 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:35:47.605885 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:47.606068 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:35:47.606510 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.025(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:35:47.606759 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:35:48.607866 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:35:48.608961 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.622094 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.622368 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[42ec049d-eaec-4617-9824-4c1df5a0460e]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:35:48.623833 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.627032 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.639235 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.639471 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[05427579-c551-4ac6-b525-bb7d715c2d46]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.639869 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.640767 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.656441 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.015s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.656664 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[119e9ddb-1d47-4b58-81bc-68d722a7064e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.656964 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.657809 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v WxNiQ5U4NP3SPA2nmzjf {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.671377 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v WxNiQ5U4NP3SPA2nmzjf" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.671558 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fed31303-76a8-4b9b-8423-01ecdfdff797]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.671913 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'WxNiQ5U4NP3SPA2nmzjf'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.673163 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.688014 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.688447 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d129bb9a-830b-43f2-8a5d-5493024e2703]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.688880 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.689740 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.705461 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.015s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.705878 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f254759f-6cd2-4303-acae-545bd540db6c]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.706705 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:35:48.706705 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:35:48.707014 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:35:48.707014 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:35:48.708405 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.725679 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --login" returned: 0 in 0.017s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.726086 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f19f3d5-a8b3-4207-80c3-47ee54a4bbd6]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.727246 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93, portal: 38.102.83.58,3260] successful. Nov 25 13:35:48.727246 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.728012 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.737961 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.737961 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebbb53af-d8f3-4c63-b0d8-0d55e88e29cf]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.737961 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:35:48.738403 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.755993 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.756266 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8dc2f4ae-d868-4692-87d4-caab114faa3b]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\ntcp: [29] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.756663 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:35:48.756663 np0005535648.novalocal cinder-backup[107475]: tcp: [29] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 (non-flash) Nov 25 13:35:48.756663 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:35:48.756806 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:35:48.756806 np0005535648.novalocal cinder-backup[107475]: tcp: [29] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 (non-flash) Nov 25 13:35:48.756806 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:35:48.757155 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.177s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:35:48.757369 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:35:48.757940 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] HCTL ('3', '-', '-', 0) found on session 29 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:35:48.758116 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:35:48.758940 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:48.789700 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.029s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:48.789911 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcb23515-515c-4e92-9ca9-bc46a288e7bb]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:35:48.790635 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 29 and hctl ['3', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:35:49.792668 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 29 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:35:49.793404 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93', 'target_lun': 0, 'volume_id': '4db5b17c-8deb-40e5-97e5-5e220df30a93', 'auth_method': 'CHAP', 'auth_username': 'WxNiQ5U4NP3SPA2nmzjf', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:35:49.794375 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.231s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:35:49.794957 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2231ms) {'type': 'block', 'scsi_wwn': '3600140575b35cd4c1784754bfa5a9e4d', 'path': '/dev/sdb'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:35:49.795614 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:35:49.821773 np0005535648.novalocal sudo[150852]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 25 13:35:50.181242 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.385s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:35:50.187301 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:35:50.194281 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] _create_container started, container: tempest-volumesbackupstest-backup-container-1326658444,backup: 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:35:50.315498 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] generate_object_name_prefix: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:35:50.320373 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] starting backup of volume: 6fd4cbf1-dd63-4003-908c-0042894cc452, volume size: 1073741824, object names prefix volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:35:50.510506 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:50.646296 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:50.646409 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:50.765570 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:50.765892 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:51.043689 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:51.156675 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:51.156675 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:51.278421 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:51.278421 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:51.914011 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:52.068653 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:52.069417 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:52.222143 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:52.222307 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:52.379035 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:52.507717 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:52.507862 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:52.655176 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:52.655470 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:53.047221 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:53.179151 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:53.179151 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:53.344268 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:53.344268 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:53.513584 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:53.628290 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:53.628475 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:53.765813 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:53.766014 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:54.518332 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:54.691010 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:54.691184 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:54.851767 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:54.851996 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:55.239472 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:55.372657 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:55.372899 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:55.551395 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:55.552087 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:55.898206 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:56.059324 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:56.059324 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:56.194018 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:56.194388 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:56.751969 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:56.851547 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:56.851732 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:56.976889 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:56.977162 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:57.137732 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:57.270723 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:57.270723 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:57.434245 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:57.434561 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:57.635976 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:57.753595 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:57.754101 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:57.895076 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:57.895232 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:58.107147 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:58.240819 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:58.241086 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:58.383669 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:58.383669 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:58.541297 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:58.647050 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:58.647050 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:58.768773 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:58.769378 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:58.942624 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:59.080908 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:59.081320 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:59.196630 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:59.196762 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:59.317856 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:59.434853 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:59.435434 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:59.561849 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:59.562249 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:35:59.749105 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:35:59.864404 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:35:59.864911 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:35:59.980145 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:35:59.980422 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:36:00.130594 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:36:00.296275 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:36:00.296275 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:36:00.414695 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:36:00.414955 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:36:00.601304 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:36:00.787644 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:36:00.787644 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:36:00.970264 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:36:00.970794 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:36:01.566743 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:36:01.703495 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:36:01.703795 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:36:01.838316 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:36:01.838730 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:36:01.988958 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:36:02.078629 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:36:02.078775 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:36:02.183416 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup MD5 for volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:36:02.183853 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:36:02.190413 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:36:02.199769 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:36:02.199960 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:36:02.203650 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:36:02.203837 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:36:02.207573 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:36:02.207842 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-1326658444, sha256file filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:36:02.284274 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:36:02.285121 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-1326658444, metadata filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:36:02.324234 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "0afbd6a6-8e37-4526-a7c6-18ac3231ae7d",\n "backup_name": "tempest-VolumesBackupsTest-Backup-2042152272",\n "created_at": "2025-11-25 18:35:42+00:00",\n "objects": [\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "6fd4cbf1-dd63-4003-908c-0042894cc452",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"6fd4cbf1-dd63-4003-908c-0042894cc452\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"39069811c8de4688bdc78baf4cf5dc3b\\", \\"project_id\\": \\"8ad212ee97684fec9bf2f2dc25f47549\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:34:49.000000\\", \\"launched_at\\": \\"2025-11-25T18:34:50.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1280292458\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 0\\", \\"provider_auth\\": \\"CHAP e8viibNK6HqLMZ6qfWDC fynrFKHpGv47AHai\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:34:49.000000\\", \\"updated_at\\": \\"2025-11-25T18:35:42.000000\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:36:02.327772 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] backup 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:36:02.403007 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.421479 np0005535648.novalocal sudo[150878]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 25 13:36:02.742898 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.339s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.744967 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93', 'target_lun': 0, 'volume_id': '4db5b17c-8deb-40e5-97e5-5e220df30a93', 'auth_method': 'CHAP', 'auth_username': 'WxNiQ5U4NP3SPA2nmzjf', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '3600140575b35cd4c1784754bfa5a9e4d', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:02.745413 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:02.746389 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:02.748173 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.756049 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.756473 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2000baa6-7da8-4a10-be15-596ccebba709]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:02.757183 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:02.757584 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:36:02.757584 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:36:02.757584 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:36:02.758096 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:36:02.758997 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.764234 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.005s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.764596 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[57c5bbb0-14b6-4c5f-99ad-a4687410d817]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93\n38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.765626 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.772232 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.006s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.772667 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b23592a-db25-45bd-8ccc-784410f9fc00]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\ntcp: [29] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.773291 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:02.773291 np0005535648.novalocal cinder-backup[107475]: tcp: [29] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 (non-flash) Nov 25 13:36:02.773291 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:02.773711 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:02.773711 np0005535648.novalocal cinder-backup[107475]: tcp: [29] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 (non-flash) Nov 25 13:36:02.773711 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:02.774988 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bd9ab93a0>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93'): ({'sdb'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:36:02.775565 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sdb {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:36:02.776521 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.782609 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ef0184bd-795c-4024-ace5-9d53f6d055ec]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:36:02.783483 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:36:02.784272 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef0184bd-795c-4024-ace5-9d53f6d055ec]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.785025 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sdb {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:36:02.785984 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.795018 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.795338 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[84fc70ca-b243-4cad-ad25-20fdc268d61d]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.795865 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:36:02.796768 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.866561 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.069s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.866962 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d82bd66f-ce4c-439a-a80c-62b3ed5339ff]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.867697 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:36:02.868106 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:36:02.869062 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:36:02.870324 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.878835 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.879103 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[781b7924-6076-45dc-bc65-6af08531de3e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.879627 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:02.880754 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.914682 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --logout" returned: 0 in 0.034s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.914985 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ee1d7f5-e8ea-4122-9d2c-f8a00541b6a4]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93, portal: 38.102.83.58,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.915619 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93, portal: 38.102.83.58,3260] Nov 25 13:36:02.915619 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93, portal: 38.102.83.58,3260] successful. Nov 25 13:36:02.915619 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:02.916512 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:02.924667 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-4db5b17c-8deb-40e5-97e5-5e220df30a93 -p 38.102.83.58:3260 --op delete" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:02.924930 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[194c568c-2e15-44d0-8248-8c4923a124f6]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:02.925795 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:02.925795 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.179s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:02.926002 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (181ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:06.470299 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-6598e729-777c-4977-a08e-9ee941ba5580 tempest-VolumesBackupsTest-210128247 None] Create backup finished. backup: 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d. Nov 25 13:36:07.362501 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Create backup started, backup: 4014e9a7-8521-4dee-b7ab-1d6acbf99030 volume: 6fd4cbf1-dd63-4003-908c-0042894cc452. Nov 25 13:36:07.366488 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container=None,created_at=2025-11-25T18:36:07Z,data_timestamp=2025-11-25T18:36:07Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=4014e9a7-8521-4dee-b7ab-1d6acbf99030,metadata={},num_dependent_backups=0,object_count=0,parent=Backup(0afbd6a6-8e37-4526-a7c6-18ac3231ae7d),parent_id='0afbd6a6-8e37-4526-a7c6-18ac3231ae7d',project_id='8ad212ee97684fec9bf2f2dc25f47549',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='39069811c8de4688bdc78baf4cf5dc3b',volume_id=6fd4cbf1-dd63-4003-908c-0042894cc452) Nov 25 13:36:08.844815 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:36:08.845659 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:36:08.845921 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:08.847605 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:08.855697 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:08.855993 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[555abadb-2732-4bd2-b447-41dd43a27922]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[a37f1a03-0452-4ecf-8595-3fb6e4b104c9]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:08.858415 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:36:08.859006 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a37f1a03-0452-4ecf-8595-3fb6e4b104c9]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:08.859399 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:36:08.859926 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:36:08.860990 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:08.869746 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:08.869975 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c991334b-a3c0-4619-b4b8-0cbb0779d314]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:08.870857 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:08.879481 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:08.879692 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4233310-abb1-4f01-915c-f4a5b6e98f5f]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:08.881090 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a60aed30-36b7-4e74-98ae-b51762dbf6ef]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:08.881757 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:08.894650 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:08.895072 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:36:08.898906 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:36:08.900051 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fef2:8c0b', 'fe80::bc21:6dff:fe62:75c9'] Nov 25 13:36:08.900321 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:36:08.900633 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:36:08.901176 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (55ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fef2:8c0b', 'fe80::bc21:6dff:fe62:75c9']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:12.912601 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:36:12.913387 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b', 'target_lun': 0, 'volume_id': 'e55203b7-6839-40b7-a171-c9fa661fb91b', 'auth_method': 'CHAP', 'auth_username': 'PnNtj3TquD9oyV6Ue5TL', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:12.913595 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:12.914093 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:12.914286 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b', 'target_lun': 0, 'volume_id': 'e55203b7-6839-40b7-a171-c9fa661fb91b', 'auth_method': 'CHAP', 'auth_username': 'PnNtj3TquD9oyV6Ue5TL', 'auth_password': 'QnevmpRbwWe82Ch4', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:36:12.915646 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:12.925004 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[d67f246c-799d-47ed-b536-50cf5350a5e2]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:36:12.927741 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:36:12.929789 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[d67f246c-799d-47ed-b536-50cf5350a5e2]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:12.930848 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:12.931725 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:12.932116 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:36:12.933365 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:12.942806 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260" returned: 21 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:12.943096 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[796071e3-60ec-431c-aaa3-0aec9e46a673]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:12.943687 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:36:12.943687 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:12.944947 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:12.954133 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:12.954349 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c062f12f-69f1-463e-9f7b-b45cad3a5297]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:12.954851 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b] added Nov 25 13:36:12.954851 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:36:12.954851 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:36:12.954851 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:36:12.954851 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:12.955041 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:36:12.955625 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.023(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:36:12.955866 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:36:13.957898 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:36:13.960028 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:13.971418 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:13.971670 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[5701b184-5a63-4f58-abb8-8f3271b25589]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:36:13.972904 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:13.976714 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:13.989323 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.015s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:13.989740 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[59ab43ca-a1d8-4bd3-8e78-a187b1b60ea7]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:13.990471 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:13.991883 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.007141 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.015s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.007503 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[81607245-53d6-4baa-a9df-7668c10983de]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.008173 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:14.009603 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.auth.username -v PnNtj3TquD9oyV6Ue5TL {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.025895 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.auth.username -v PnNtj3TquD9oyV6Ue5TL" returned: 0 in 0.016s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.026282 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc644fd7-4a4e-4cc5-b7a9-27b6bb5c667e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.027010 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'PnNtj3TquD9oyV6Ue5TL'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:14.028509 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.046194 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.046512 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ad2478b-1ccc-440f-a324-f6b43def4412]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.047412 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:14.048621 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.059185 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.059385 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6581e5e-7523-46cc-8350-576c94a51cae]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.059896 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:14.059896 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:14.060202 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:14.060202 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:14.061488 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.080632 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --login" returned: 0 in 0.019s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.081383 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[dff1ed11-e9ab-4884-9f90-d75f2b943ca6]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.082469 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b, portal: 38.102.83.58,3260] successful. Nov 25 13:36:14.082469 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:14.086045 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.098932 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.015s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.099187 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7349db60-71c9-4150-8c00-26fd212e9111]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.099774 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:14.100748 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.118098 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.016s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.118359 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fb4e968c-9b67-41e6-a2c1-e436fa5b4d72]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\ntcp: [30] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.119205 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:14.119205 np0005535648.novalocal cinder-backup[107475]: tcp: [30] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b (non-flash) Nov 25 13:36:14.119205 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:14.119463 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:14.119463 np0005535648.novalocal cinder-backup[107475]: tcp: [30] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b (non-flash) Nov 25 13:36:14.119463 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:14.120093 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.188s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:14.120357 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:36:14.121261 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] HCTL ('3', '-', '-', 0) found on session 30 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:36:14.121584 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:36:14.122710 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:14.150176 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.026s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:14.150520 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[45b1a719-9800-47ed-9e0a-9c3e51424140]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:14.151963 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 30 and hctl ['3', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:36:15.154090 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 30 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:36:15.154953 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b', 'target_lun': 0, 'volume_id': 'e55203b7-6839-40b7-a171-c9fa661fb91b', 'auth_method': 'CHAP', 'auth_username': 'PnNtj3TquD9oyV6Ue5TL', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:36:15.155933 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.242s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:15.156425 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2242ms) {'type': 'block', 'scsi_wwn': '360014058daf0ec6977b4646b0d8deb85', 'path': '/dev/sdb'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:15.157175 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:15.181185 np0005535648.novalocal sudo[151097]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 25 13:36:15.573771 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.416s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:15.579737 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:36:15.597996 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _read_sha256file started, container name: tempest-volumesbackupstest-backup-container-1326658444, sha256 filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_sha256file. {{(pid=107475) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:336}} Nov 25 13:36:15.647055 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _read_sha256file finished. {{(pid=107475) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:343}} Nov 25 13:36:15.654993 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _create_container started, container: volumebackups,backup: 4014e9a7-8521-4dee-b7ab-1d6acbf99030. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:36:15.811439 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] generate_object_name_prefix: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030 {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:36:15.825223 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] starting backup of volume: 6fd4cbf1-dd63-4003-908c-0042894cc452, volume size: 1073741824, object names prefix volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:36:22.578379 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:36:22.586673 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:36:22.586970 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:36:22.591909 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:36:22.592165 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:36:22.595807 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:36:22.596254 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _write_sha256file started, container name: volumebackups, sha256file filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:36:22.694425 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:36:22.695172 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _write_metadata started, container name: volumebackups, metadata filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:36:22.735647 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "4014e9a7-8521-4dee-b7ab-1d6acbf99030",\n "backup_name": null,\n "created_at": "2025-11-25 18:36:07+00:00",\n "objects": [],\n "parent_id": "0afbd6a6-8e37-4526-a7c6-18ac3231ae7d",\n "version": "1.0.0",\n "volume_id": "6fd4cbf1-dd63-4003-908c-0042894cc452",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"6fd4cbf1-dd63-4003-908c-0042894cc452\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"39069811c8de4688bdc78baf4cf5dc3b\\", \\"project_id\\": \\"8ad212ee97684fec9bf2f2dc25f47549\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:34:49.000000\\", \\"launched_at\\": \\"2025-11-25T18:34:50.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1280292458\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 0\\", \\"provider_auth\\": \\"CHAP e8viibNK6HqLMZ6qfWDC fynrFKHpGv47AHai\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:34:49.000000\\", \\"updated_at\\": \\"2025-11-25T18:36:07.000000\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:36:22.740056 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] backup 4014e9a7-8521-4dee-b7ab-1d6acbf99030 finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:36:22.813425 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:22.843265 np0005535648.novalocal sudo[151106]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 25 13:36:23.276690 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.463s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.278802 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b', 'target_lun': 0, 'volume_id': 'e55203b7-6839-40b7-a171-c9fa661fb91b', 'auth_method': 'CHAP', 'auth_username': 'PnNtj3TquD9oyV6Ue5TL', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014058daf0ec6977b4646b0d8deb85', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:23.279106 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:23.280007 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:23.282249 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.296445 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.296748 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd78958f-8f90-426f-a142-e03b16530f7b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:23.297263 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:23.297663 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:36:23.297663 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:36:23.297663 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:36:23.298158 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:36:23.299128 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.309269 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.309632 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[94272b5c-01f6-4ff5-84ca-89fe65a0f888]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452\n38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.310835 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.323460 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.323763 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f133547-9864-48ed-bfde-7ef9310040e1]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\ntcp: [30] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.324222 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:23.324222 np0005535648.novalocal cinder-backup[107475]: tcp: [30] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b (non-flash) Nov 25 13:36:23.324222 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:23.324494 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:23.324494 np0005535648.novalocal cinder-backup[107475]: tcp: [30] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b (non-flash) Nov 25 13:36:23.324494 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:23.325610 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bdca01580>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b'): ({'sdb'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:36:23.326135 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sdb {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:36:23.327130 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.333706 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[ae9e3318-70e2-4fa9-80c6-c48b073d5c14]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:36:23.336047 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:36:23.337970 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae9e3318-70e2-4fa9-80c6-c48b073d5c14]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.338730 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sdb {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:36:23.339673 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.347452 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.347908 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ce6f7fd-69a0-4758-b90b-4b32c3480178]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.348238 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:36:23.349395 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.417128 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.067s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.417372 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3266bb38-acf1-430f-aad2-6ddebf6ac2e2]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.418176 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:36:23.418824 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:36:23.419667 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:36:23.421128 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.436639 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.015s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.436987 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f906511-5c55-44ec-8780-254c4cfcb79f]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.438135 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:23.439387 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.476340 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --logout" returned: 0 in 0.037s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.476528 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[93b381cd-f63e-451b-9c0a-077ca589b7fb]: (4, ('Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b, portal: 38.102.83.58,3260]\nLogout of [sid: 30, target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.477164 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 30, target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b, portal: 38.102.83.58,3260] Nov 25 13:36:23.477164 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 30, target: iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b, portal: 38.102.83.58,3260] successful. Nov 25 13:36:23.477164 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:23.478220 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:23.488030 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-e55203b7-6839-40b7-a171-c9fa661fb91b -p 38.102.83.58:3260 --op delete" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:23.488290 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[60f16c73-280b-4a1d-a4d5-cce0a2b7bde2]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:23.488910 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:23.489364 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.209s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:23.489673 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (210ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:27.045610 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-4db7e76f-e690-490f-bfd9-d8f12b04fe88 tempest-VolumesBackupsTest-210128247 None] Create backup finished. backup: 4014e9a7-8521-4dee-b7ab-1d6acbf99030. Nov 25 13:36:28.070994 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] Delete backup started, backup: 4014e9a7-8521-4dee-b7ab-1d6acbf99030. Nov 25 13:36:28.072731 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:36:28.072875 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:36:28.073165 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] delete started, backup: 4014e9a7-8521-4dee-b7ab-1d6acbf99030, container: volumebackups, prefix: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:36:28.126799 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030_metadata', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:36:28.155366 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030_metadata in container: volumebackups. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:28.191668 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183615/az_nova_backup_4014e9a7-8521-4dee-b7ab-1d6acbf99030_sha256file in container: volumebackups. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:28.191668 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] delete 4014e9a7-8521-4dee-b7ab-1d6acbf99030 finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:36:28.196880 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:36:28.216261 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] Created reservations ['09331db0-a1cd-45e6-a2a7-efa195802024', '9020756e-0f5e-4079-8946-0c5020e184c4'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:36:28.249904 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-2de17622-2f75-470f-b00f-6376b0401bb3 tempest-VolumesBackupsTest-210128247 None] Delete backup finished, backup 4014e9a7-8521-4dee-b7ab-1d6acbf99030 deleted. Nov 25 13:36:29.245823 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Create backup started, backup: 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44 volume: 6fd4cbf1-dd63-4003-908c-0042894cc452. Nov 25 13:36:29.250661 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupstest-backup-container-275375754',created_at=2025-11-25T18:36:29Z,data_timestamp=2025-11-25T18:36:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1220350270',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=68c5eafe-053e-4cf0-bfb8-0306e9cc2e44,metadata={},num_dependent_backups=0,object_count=0,parent=Backup(0afbd6a6-8e37-4526-a7c6-18ac3231ae7d),parent_id='0afbd6a6-8e37-4526-a7c6-18ac3231ae7d',project_id='8ad212ee97684fec9bf2f2dc25f47549',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='39069811c8de4688bdc78baf4cf5dc3b',volume_id=6fd4cbf1-dd63-4003-908c-0042894cc452) Nov 25 13:36:30.844649 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:36:30.845192 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:36:30.845263 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:30.846639 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:30.855896 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:30.856109 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8967c865-4dc5-4eb5-b701-e96496037405]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[f57a3463-6ceb-405e-83ed-d3cac6a7f60f]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:30.858149 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:36:30.858815 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f57a3463-6ceb-405e-83ed-d3cac6a7f60f]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:30.859286 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:36:30.859647 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:36:30.860391 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:30.871444 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:30.871992 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a45bd803-902d-4157-a0e2-e6ab2281e16d]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:30.872802 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:30.884162 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:30.884511 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[6701e0d7-b6ae-4005-a7ad-d54dc3e66122]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:30.885428 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd8bea00-c452-4da9-9bda-6c9c0263635f]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:30.885817 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:30.894930 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:30.895215 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:36:30.896445 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:36:30.897121 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fef2:8c0b', 'fe80::bc21:6dff:fe62:75c9'] Nov 25 13:36:30.897288 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:36:30.897443 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:36:30.897787 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] <== get_connector_properties: return (51ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40', 'fe80::fc16:3eff:fef2:8c0b', 'fe80::bc21:6dff:fe62:75c9']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:34.406286 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:36:34.407525 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243', 'target_lun': 0, 'volume_id': 'ea22ead0-3abe-45df-bc06-480bbf48f243', 'auth_method': 'CHAP', 'auth_username': 'CVoSZM6Bfr7uHkKUH3Uj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:34.407933 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:34.408562 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:34.408887 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243', 'target_lun': 0, 'volume_id': 'ea22ead0-3abe-45df-bc06-480bbf48f243', 'auth_method': 'CHAP', 'auth_username': 'CVoSZM6Bfr7uHkKUH3Uj', 'auth_password': '9Pzyipjz8nxTvMQU', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:36:34.409995 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:34.415053 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[02cb68e1-8d68-43af-8149-7b74def62e01]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:36:34.416081 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:36:34.417381 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[02cb68e1-8d68-43af-8149-7b74def62e01]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:34.417909 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:34.418628 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:34.419081 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:36:34.420053 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:34.429476 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260" returned: 21 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:34.429840 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[de3b94a8-f345-4c4b-bc21-86c90c9d6117]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:34.430386 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:36:34.430386 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:34.431445 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:34.441096 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:34.441511 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[18a2e5f8-de8f-4f4f-a6c1-bfebc5418890]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:34.442106 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243] added Nov 25 13:36:34.442106 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:36:34.442106 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:36:34.442106 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:36:34.442106 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:34.442476 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:36:34.443184 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.024(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:36:34.443601 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:36:34.627915 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:36:34.628258 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:36:35.445269 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:36:35.446585 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.457739 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.458053 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6b891b5-06ef-47d0-a53c-9e70940365f3]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:36:35.460037 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.462833 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.472655 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.472914 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[58e97603-a4c0-413e-abc6-b781f7a6c8d7]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.473399 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.474374 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.486136 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.486386 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b79b5c86-7286-4af1-bd17-d15ac0c2984e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.486889 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.487876 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v CVoSZM6Bfr7uHkKUH3Uj {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.500201 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.auth.username -v CVoSZM6Bfr7uHkKUH3Uj" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.500471 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbd9238e-ae24-4f80-958d-18d0bcd91c9f]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.500977 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CVoSZM6Bfr7uHkKUH3Uj'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.502092 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.513510 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.513823 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[70a44747-ad46-49cb-a1e7-f0d224d9985e]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.514444 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.515560 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.528683 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.528927 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[00f21fb4-f57a-4a6e-81a0-bbb02c620b32]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.529381 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:35.529381 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:35.529678 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:35.529678 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:35.530770 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.551605 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --login" returned: 0 in 0.020s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.551915 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[77742083-ec83-447d-bd90-2221add7365d]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.552686 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243, portal: 38.102.83.58,3260] successful. Nov 25 13:36:35.552686 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.554040 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.567000 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.567231 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f93e9a71-ae5e-435f-b93f-15009a701d72]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.567814 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:35.568852 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.582848 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.014s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.583107 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a85847de-3b01-40ab-8131-22c805bd2012]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\ntcp: [31] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.583705 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:35.583705 np0005535648.novalocal cinder-backup[107475]: tcp: [31] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 (non-flash) Nov 25 13:36:35.583705 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:35.583997 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:35.583997 np0005535648.novalocal cinder-backup[107475]: tcp: [31] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 (non-flash) Nov 25 13:36:35.583997 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:35.584529 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.166s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:35.584915 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:36:35.585770 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] HCTL ('3', '-', '-', 0) found on session 31 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:36:35.586125 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Scanning host 3 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:36:35.587183 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host3/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:35.611836 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host3/scan" returned: 0 in 0.024s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:35.612188 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[da693c89-ba50-464c-b159-c1ecbaad7276]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:35.613065 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 31 and hctl ['3', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:36:36.614504 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Searching for a device in session 31 and hctl ['3', '*', '*', 0] yield: sdb {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:36:36.615258 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Connected to sdb using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243', 'target_lun': 0, 'volume_id': 'ea22ead0-3abe-45df-bc06-480bbf48f243', 'auth_method': 'CHAP', 'auth_username': 'CVoSZM6Bfr7uHkKUH3Uj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:36:36.615715 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.207s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:36.615962 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] <== connect_volume: return (2208ms) {'type': 'block', 'scsi_wwn': '360014054e4dc38490f24172bea772429', 'path': '/dev/sdb'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:36.616668 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:36.643681 np0005535648.novalocal sudo[151357]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb Nov 25 13:36:37.024592 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sdb" returned: 0 in 0.408s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:37.031555 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:36:37.034305 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=83,cinder:UPDATE=22,cinder:INSERT=2 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:36:37.036380 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _read_sha256file started, container name: tempest-volumesbackupstest-backup-container-1326658444, sha256 filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_sha256file. {{(pid=107475) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:336}} Nov 25 13:36:37.080624 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _read_sha256file finished. {{(pid=107475) _read_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:343}} Nov 25 13:36:37.090261 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _create_container started, container: tempest-volumesbackupstest-backup-container-275375754,backup: 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:36:37.194732 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] generate_object_name_prefix: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44 {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:36:37.203944 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] starting backup of volume: 6fd4cbf1-dd63-4003-908c-0042894cc452, volume size: 1073741824, object names prefix volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:36:43.394413 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:36:43.402427 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:36:43.402964 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:36:43.407544 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:36:43.407842 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:36:43.412022 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:36:43.412469 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _write_sha256file started, container name: tempest-volumesbackupstest-backup-container-275375754, sha256file filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:36:43.504749 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:36:43.505993 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _write_metadata started, container name: tempest-volumesbackupstest-backup-container-275375754, metadata filename: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:36:43.540133 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "68c5eafe-053e-4cf0-bfb8-0306e9cc2e44",\n "backup_name": "tempest-VolumesBackupsTest-Backup-1220350270",\n "created_at": "2025-11-25 18:36:29+00:00",\n "objects": [],\n "parent_id": "0afbd6a6-8e37-4526-a7c6-18ac3231ae7d",\n "version": "1.0.0",\n "volume_id": "6fd4cbf1-dd63-4003-908c-0042894cc452",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"6fd4cbf1-dd63-4003-908c-0042894cc452\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"39069811c8de4688bdc78baf4cf5dc3b\\", \\"project_id\\": \\"8ad212ee97684fec9bf2f2dc25f47549\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"attached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:34:49.000000\\", \\"launched_at\\": \\"2025-11-25T18:34:50.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsTest-Volume-1280292458\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 0\\", \\"provider_auth\\": \\"CHAP e8viibNK6HqLMZ6qfWDC fynrFKHpGv47AHai\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"in-use\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:34:49.000000\\", \\"updated_at\\": \\"2025-11-25T18:36:29.000000\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:36:43.544792 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] backup 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44 finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:36:43.645982 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:43.663909 np0005535648.novalocal sudo[151365]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb Nov 25 13:36:43.983857 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sdb" returned: 0 in 0.338s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:43.985610 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243', 'target_lun': 0, 'volume_id': 'ea22ead0-3abe-45df-bc06-480bbf48f243', 'auth_method': 'CHAP', 'auth_username': 'CVoSZM6Bfr7uHkKUH3Uj', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014054e4dc38490f24172bea772429', 'path': '/dev/sdb'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:36:43.985945 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:36:43.986726 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:36:43.988437 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:43.996968 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:43.997274 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[44bd4bcc-2665-4168-a1b5-ea20497ceb47]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:36:43.997827 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:43.998244 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:36:43.998244 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:36:43.998244 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:36:43.998605 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:36:43.999592 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.007114 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.007498 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[6986c9d8-b47f-4bfa-b9c7-a667aa27fbce]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452\n38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.008322 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.021812 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.022067 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[80bfe71e-3419-42a8-9fd1-77d07c85af79]: (4, ('tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash)\ntcp: [31] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.022338 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('-m', 'session'): stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:44.022338 np0005535648.novalocal cinder-backup[107475]: tcp: [31] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 (non-flash) Nov 25 13:36:44.022338 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:36:44.022499 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsi session list stdout=tcp: [28] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-6fd4cbf1-dd63-4003-908c-0042894cc452 (non-flash) Nov 25 13:36:44.022499 np0005535648.novalocal cinder-backup[107475]: tcp: [31] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 (non-flash) Nov 25 13:36:44.022499 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:36:44.023023 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Resulting device map defaultdict(. at 0x7f9bdccb2020>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243'): ({'sdb'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:36:44.023231 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Removing single pathed devices sdb {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:36:44.024016 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.029988 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[2608b535-d348-4871-8af1-aa529127cc75]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:36:44.032526 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:36:44.034284 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2608b535-d348-4871-8af1-aa529127cc75]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.035121 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Flushing IO for device /dev/sdb {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:36:44.035878 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.044572 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.044872 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2903a8f-6773-467f-bd9c-5742e41c8209]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.045358 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:36:44.046327 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.107605 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.061s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.107906 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7327696a-072c-4d64-917a-8fced77fd9ae]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.108451 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Checking to see if SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:36:44.108802 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] SCSI volumes sdb have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:36:44.109488 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:36:44.110378 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.120743 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.010s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.121610 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e31f369-d5c8-4d14-a244-67d75cc08915]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.122162 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:44.123354 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.158170 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --logout" returned: 0 in 0.035s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.158344 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[44889d83-97f9-4098-8fbd-cc8224f25fb7]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243, portal: 38.102.83.58,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.158697 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243, portal: 38.102.83.58,3260] Nov 25 13:36:44.158697 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243, portal: 38.102.83.58,3260] successful. Nov 25 13:36:44.158697 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:44.159292 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:36:44.165570 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-ea22ead0-3abe-45df-bc06-480bbf48f243 -p 38.102.83.58:3260 --op delete" returned: 0 in 0.006s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:36:44.165764 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[8268115d-fee3-47d1-8e22-699e3f5a50ea]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:36:44.165984 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:36:44.166196 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.180s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:36:44.166368 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] <== disconnect_volume: return (180ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:36:47.716948 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-8c60a75a-ca43-4283-97e1-f0833dbd813b tempest-VolumesBackupsTest-210128247 None] Create backup finished. backup: 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44. Nov 25 13:36:47.861885 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] Delete backup started, backup: 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44. Nov 25 13:36:47.864360 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:36:47.864812 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:36:47.865225 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] delete started, backup: 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44, container: tempest-volumesbackupstest-backup-container-275375754, prefix: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:36:47.944634 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44_metadata', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:36:47.978590 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44_metadata in container: tempest-volumesbackupstest-backup-container-275375754. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:48.010745 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183637/az_nova_backup_68c5eafe-053e-4cf0-bfb8-0306e9cc2e44_sha256file in container: tempest-volumesbackupstest-backup-container-275375754. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:48.011111 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] delete 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44 finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:36:48.016794 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:36:48.041730 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] Created reservations ['afb812b1-5704-482f-956e-3750489cd24f', '3e6d58e7-130f-4271-987b-0a3ec13fa666'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:36:48.082226 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-817f487e-cd6c-4391-b105-e438422b0bd7 tempest-VolumesBackupsTest-210128247 None] Delete backup finished, backup 68c5eafe-053e-4cf0-bfb8-0306e9cc2e44 deleted. Nov 25 13:36:49.049836 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] Delete backup started, backup: 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d. Nov 25 13:36:49.050980 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_8ad212ee97684fec9bf2f2dc25f47549 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:36:49.051192 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:36:49.051448 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] delete started, backup: 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d, container: tempest-volumesbackupstest-backup-container-1326658444, prefix: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:36:49.107895 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] generated object list: ['volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00001', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00002', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00003', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00004', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00005', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00006', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00007', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00008', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00009', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00010', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00011', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00012', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00013', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00014', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00015', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00016', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00017', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00018', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00019', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00020', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00021', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_metadata', 'volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:36:49.134251 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00001 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.162190 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00002 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.195865 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00003 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.222182 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00004 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.247565 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00005 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.269932 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00006 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.289853 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00007 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.311595 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00008 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.335941 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00009 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.366218 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00010 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.398427 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00011 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.427348 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00012 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.458325 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00013 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.483961 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00014 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.528343 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00015 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.559497 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00016 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.586034 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00017 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.611275 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00018 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.637529 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00019 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.664477 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00020 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.689157 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d-00021 in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.718602 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_metadata in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.758912 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] deleted object: volume_6fd4cbf1-dd63-4003-908c-0042894cc452/20251125183550/az_nova_backup_0afbd6a6-8e37-4526-a7c6-18ac3231ae7d_sha256file in container: tempest-volumesbackupstest-backup-container-1326658444. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:36:49.759275 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] delete 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:36:49.763482 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:36:49.780911 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] Created reservations ['8b47ff7d-555b-48a6-826c-a9a434159048', '246a3cbc-2a65-4d3d-b587-ddf23e32ccb1'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:36:49.800099 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-8f4d2211-4eeb-4e9d-bc59-79cef317e0be tempest-VolumesBackupsTest-210128247 None] Delete backup finished, backup 0afbd6a6-8e37-4526-a7c6-18ac3231ae7d deleted. Nov 25 13:36:59.798697 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=46,cinder:UPDATE=19,cinder:INSERT=4 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:37:09.850720 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Create backup started, backup: 81f1d77a-5ac7-44bb-bf50-ae172ce21df5 volume: d41e9df7-0b74-4c59-84e6-0e3eac93d37f. Nov 25 13:37:09.855626 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Call Volume Manager to get_backup_device for Backup(availability_zone='nova',container='tempest-volumesbackupsv39test-backup-container-353768439',created_at=2025-11-25T18:37:09Z,data_timestamp=2025-11-25T18:37:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-477837502',encryption_key_id=None,fail_reason=None,host='np0005535648.novalocal',id=81f1d77a-5ac7-44bb-bf50-ae172ce21df5,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='ef208f8a9de94f50840218015bfcd515',restore_volume_id=None,service='cinder.backup.drivers.swift.SwiftBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='e351f755cf754f40925b369d03d6ad5d',volume_id=d41e9df7-0b74-4c59-84e6-0e3eac93d37f) Nov 25 13:37:09.879446 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_ef208f8a9de94f50840218015bfcd515 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:37:09.879708 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:37:09.880133 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] ==> get_connector_properties: call "{'root_helper': 'sudo cinder-rootwrap /etc/cinder/rootwrap.conf', 'my_ip': '38.102.83.58', 'multipath': False, 'enforce_multipath': False, 'host': None, 'execute': None}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:37:09.881021 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:09.885095 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.004s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:09.885277 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0d049fd-c866-4018-bf09-52e03276c4fe]: (4, ('InitiatorName=iqn.1994-05.com.redhat:521f6b681fe5\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[0da5ad9b-6527-4e97-9861-fb8b19575ee5]: [Errno 2] No such file or directory: '/dev/scini' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: with open_scini_device() as fd: Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__ Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: return next(self.gen) Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^ Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR) Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:09.886980 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: '/dev/scini' Nov 25 13:37:09.887466 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0da5ad9b-6527-4e97-9861-fb8b19575ee5]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 57, in get_guid\n with open_scini_device() as fd:\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__\n return next(self.gen)\n ^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/scaleio.py", line 40, in open_scini_device\n fd = os.open(SCINI_DEVICE_PATH, os.O_RDWR)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: \'/dev/scini\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:09.887718 np0005535648.novalocal cinder-backup[107475]: ERROR os_brick.initiator.connectors.scaleio [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Error querying sdc guid: [Errno 2] No such file or directory: FileNotFoundError: [Errno 2] No such file or directory Nov 25 13:37:09.888062 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.scaleio [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Unable to find SDC guid: Error querying sdc guid: [Errno 2] No such file or directory Nov 25 13:37:09.888727 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:09.897299 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:09.897677 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[10bfbeba-b9ad-43f4-a051-d2aff609f87e]: (4, ('/dev/vda2\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:09.898401 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/vda2 -s UUID -o value {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:09.907915 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/vda2 -s UUID -o value" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:09.908139 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[2202e2d7-7ae4-4664-b051-81005e478ab8]: (4, ('3fc68afb-d7e4-475c-9164-ea34f338d965\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:09.909021 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[7478835c-02e8-4ca2-bb0e-dc8ff71ac6f2]: (4, '4575dca2-63e8-4855-b485-32f10c4bd17a') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:09.909567 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Running cmd (subprocess): nvme version {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:09.921838 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] 'nvme version' failed. Not Retrying. {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:37:09.922204 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.nvmeof [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] nvme not present on system {{(pid=107475) nvme_present /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/nvmeof.py:784}} Nov 25 13:37:09.924573 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=107475) find_dsc /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:132}} Nov 25 13:37:09.925818 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.lightos [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Current host hostNQN nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e and IP(s) are ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40'] Nov 25 13:37:09.926193 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] LIGHTOS: did not find dsc, continuing anyway. {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:109}} Nov 25 13:37:09.926584 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.lightos [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] LIGHTOS: finally hostnqn: nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e dsc: {{(pid=107475) get_connector_properties /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/lightos.py:112}} Nov 25 13:37:09.927348 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] <== get_connector_properties: return (46ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '38.102.83.58', 'host': 'np0005535648.novalocal', 'multipath': False, 'enforce_multipath': False, 'initiator': 'iqn.1994-05.com.redhat:521f6b681fe5', 'do_local_attach': False, 'uuid': '3fc68afb-d7e4-475c-9164-ea34f338d965', 'system uuid': '4575dca2-63e8-4855-b485-32f10c4bd17a', 'nvme_native_multipath': False, 'nqn': 'nqn.2014-08.org.nvmexpress:uuid:b75896e9-b5d4-4941-a6e4-36982d60c99e', 'found_dsc': '', 'host_ips': ['38.102.83.58', 'fe80::f816:3eff:fe44:96db', '192.168.122.1', '172.24.5.1', '2001:db8::2', 'fe80::b05c:acff:fe5b:3e40']} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:37:13.437368 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connector [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Factory for iscsi on None {{(pid=107475) factory /usr/local/lib/python3.12/site-packages/os_brick/initiator/connector.py:281}} Nov 25 13:37:13.438089 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] ==> connect_volume: call "{'self': , 'connection_properties': {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'target_lun': 0, 'volume_id': 'd41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'auth_method': 'CHAP', 'auth_username': 'jLBUWPmqGR66WohA4V2A', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:37:13.438259 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:37:13.438657 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.000s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:37:13.438796 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Connection properties {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'target_lun': 0, 'volume_id': 'd41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'auth_method': 'CHAP', 'auth_username': 'jLBUWPmqGR66WohA4V2A', 'auth_password': 'cJMjgFsVYsfV36VP', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False} {{(pid=107475) check_multipath /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:133}} Nov 25 13:37:13.439704 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:13.445845 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[65a3e6eb-5278-4cdd-9ba8-bece678fce30]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:37:13.448127 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:37:13.450634 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[65a3e6eb-5278-4cdd-9ba8-bece678fce30]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:13.450634 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Acquiring lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:37:13.450965 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:37:13.451136 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:37:13.451777 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:13.460895 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260" returned: 21 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:13.461029 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[44cf950b-65c0-41c6-aef7-e890cd5f9595]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:13.461314 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm (): stdout= stderr=iscsiadm: No records found Nov 25 13:37:13.461314 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:13.462013 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --interface default --op new {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:13.471222 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --interface default --op new" returned: 0 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:13.471634 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[c71bd115-d780-4ca6-9ff2-4de8e11d7761]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f] added\n', 'iscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\niscsiadm: Config file line 381 does not have value\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:13.472131 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 38.102.83.58,3260,-1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f] added Nov 25 13:37:13.472131 np0005535648.novalocal cinder-backup[107475]: stderr=iscsiadm: Config file line 381 does not have value Nov 25 13:37:13.472131 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:37:13.472131 np0005535648.novalocal cinder-backup[107475]: iscsiadm: Config file line 381 does not have value Nov 25 13:37:13.472131 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:13.472412 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Retrying to connect to iSCSI portal 38.102.83.58:3260 {{(pid=107475) _connect_to_iscsi_portal_unsafe /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1133}} Nov 25 13:37:13.473201 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Finished call to 'os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe' after 0.022(s), this was the 1st time calling it. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/after.py:44}} Nov 25 13:37:13.473565 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.utils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Retrying os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe in 1.0 seconds as it raised BrickException: An unknown exception occurred.. {{(pid=107475) log_it /usr/local/lib/python3.12/site-packages/tenacity/before_sleep.py:65}} Nov 25 13:37:14.475100 np0005535648.novalocal cinder-backup[107475]: INFO os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Trying to connect to iSCSI portal 38.102.83.58:3260 Nov 25 13:37:14.476167 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.483758 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.484026 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec89466c-1351-49e2-abc8-4f8a69b9f484]: (4, ('# BEGIN RECORD 6.2.1.11\nnode.name = iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = \niface.ipaddress = \niface.prefix_len = 0\niface.hwaddress = \niface.transport_name = tcp\niface.initiatorname = \niface.state = \niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = \niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = \niface.subnet_mask = \niface.gateway = \niface.dhcp_alt_client_id_state = \niface.dhcp_alt_client_id = \niface.dhcp_dns = \niface.dhcp_learn_iqn = \niface.dhcp_req_vendor_id_state = \niface.dhcp_vendor_id_state = \niface.dhcp_vendor_id = \niface.dhcp_slp_da = \niface.fragmentation = \niface.gratuitous_arp = \niface.incoming_forwarding = \niface.tos_state = \niface.tos = 0\niface.ttl = 0\niface.delayed_ack = \niface.tcp_nagle = \niface.tcp_wsf_state = \niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = \niface.redirect = \niface.def_task_mgmt_timeout = 0\niface.header_digest = \niface.data_digest = \niface.immediate_data = \niface.initial_r2t = \niface.data_seq_inorder = \niface.data_pdu_inorder = \niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = \niface.bidi_chap = \niface.strict_login_compliance = \niface.discovery_auth = \niface.discovery_logout = \nnode.discovery_address = \nnode.discovery_port = 0\nnode.discovery_type = fw\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = 0\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = None\nnode.session.auth.username = \nnode.session.auth.password = \nnode.session.auth.username_in = \nnode.session.auth.password_in = \nnode.session.auth.chap_algs = SHA3-256,SHA256\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = auto\nnode.session.reopen_max = 0\nnode.session.conn_reopen_log_freq = 1\nnode.conn[0].address = 38.102.83.58\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm (): stdout=# BEGIN RECORD 6.2.1.11 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.name = iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.tpgt = -1 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.startup = automatic Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.leading_login = No Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.iscsi_ifacename = default Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.net_ifacename = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.ipaddress = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.prefix_len = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.hwaddress = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.transport_name = tcp Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.initiatorname = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.vlan_id = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.vlan_priority = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.vlan_state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.iface_num = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.mtu = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.port = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.bootproto = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.subnet_mask = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.gateway = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id_state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_alt_client_id = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_dns = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_learn_iqn = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_req_vendor_id_state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id_state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_vendor_id = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.dhcp_slp_da = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.fragmentation = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.gratuitous_arp = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.incoming_forwarding = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tos_state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tos = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.ttl = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.delayed_ack = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tcp_nagle = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf_state = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tcp_wsf = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timer_scale = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.tcp_timestamp = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.redirect = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.def_task_mgmt_timeout = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.header_digest = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.data_digest = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.immediate_data = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.initial_r2t = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.data_seq_inorder = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.data_pdu_inorder = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.erl = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.max_receive_data_len = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.first_burst_len = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.max_outstanding_r2t = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.max_burst_len = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.chap_auth = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.bidi_chap = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.strict_login_compliance = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.discovery_auth = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: iface.discovery_logout = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.discovery_address = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.discovery_port = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.discovery_type = fw Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.initial_cmdsn = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.initial_login_retry_max = 8 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.xmit_thread_priority = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.cmds_max = 128 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.queue_depth = 32 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.nr_sessions = 1 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.auth.authmethod = None Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password = *** Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.auth.username_in = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.auth.password_in = Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.auth.chap_algs = SHA3-256,SHA256 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.timeo.replacement_timeout = 120 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.abort_timeout = 15 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.lu_reset_timeout = 30 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.tgt_reset_timeout = 30 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.err_timeo.host_reset_timeout = 60 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FastAbort = Yes Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.InitialR2T = No Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ImmediateData = Yes Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.FirstBurstLength = 262144 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxBurstLength = 16776192 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Retain = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.DefaultTime2Wait = 2 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxConnections = 1 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.MaxOutstandingR2T = 1 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.iscsi.ERL = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.scan = auto Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.reopen_max = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.session.conn_reopen_log_freq = 1 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].address = 38.102.83.58 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].port = 3260 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].startup = manual Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.window_size = 524288 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].tcp.type_of_service = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.logout_timeout = 15 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.login_timeout = 15 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.auth_timeout = 45 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_interval = 5 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].timeo.noop_out_timeout = 5 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.HeaderDigest = None Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.DataDigest = None Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.IFMarker = No Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: node.conn[0].iscsi.OFMarker = No Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: # END RECORD Nov 25 13:37:14.485223 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.488338 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.scan -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.498295 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.498451 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[290bd227-995f-451d-aa64-72f32be73274]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.498873 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.499532 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.507980 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.508180 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[18fee793-53c0-43b9-8d25-bb83000d8ba4]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.508527 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.509243 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.auth.username -v jLBUWPmqGR66WohA4V2A {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.520616 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.auth.username -v jLBUWPmqGR66WohA4V2A" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.520813 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[655f5ffd-0ca6-4fb9-97fd-df3517290372]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.521213 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'jLBUWPmqGR66WohA4V2A'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.522054 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.auth.password -v *** {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.535252 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.013s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.535418 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[12e10810-9b48-4947-9676-865aab13d9ba]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.535835 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.536489 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.545819 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.009s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.545990 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e51d583-3c6c-4e70-b02a-fc1bac6245e0]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.546385 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions. Nov 25 13:37:14.546385 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:37:14.546611 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsi session list stdout= stderr=iscsiadm: No active sessions. Nov 25 13:37:14.546611 np0005535648.novalocal cinder-backup[107475]: {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:37:14.546764 np0005535648.novalocal cinder-backup[107475]: WARNING os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions. Nov 25 13:37:14.546764 np0005535648.novalocal cinder-backup[107475]: Nov 25 13:37:14.547641 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --login {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.559651 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --login" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.559868 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cc45c41-da9d-4a5d-a7d1-e581ac126d5f]: (4, ('Login to [iface: default, target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.560411 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--login',): stdout=Login to [iface: default, target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f, portal: 38.102.83.58,3260] successful. Nov 25 13:37:14.560411 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.562961 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.startup -v automatic {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.573255 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.startup -v automatic" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.573472 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[78e65c4d-3f93-4613-a516-00049401e79f]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.574212 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:14.575624 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.587591 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.587892 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[84976552-015d-4cbf-982c-a56f3dc2bfb3]: (4, ('tcp: [32] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.588401 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('-m', 'session'): stdout=tcp: [32] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f (non-flash) Nov 25 13:37:14.588401 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:37:14.588700 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsi session list stdout=tcp: [32] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f (non-flash) Nov 25 13:37:14.588700 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:37:14.589252 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Lock "connect_to_iscsi_portal-38.102.83.58:3260-iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.138s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:37:14.589530 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Connected to 38.102.83.58:3260 {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:659}} Nov 25 13:37:14.590202 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] HCTL ('2', '-', '-', 0) found on session 32 with lun 0 {{(pid=107475) get_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:788}} Nov 25 13:37:14.590461 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Scanning host 2 c: -, t: -, l: 0) {{(pid=107475) scan_iscsi /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:814}} Nov 25 13:37:14.591655 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host2/scan {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:14.616178 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host2/scan" returned: 0 in 0.024s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:14.616839 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[a5e109a1-6f19-44c0-b47e-2a50ca6deebd]: (4, ('- - 0', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:14.617497 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Searching for a device in session 32 and hctl ['2', '*', '*', 0] yield: None {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:37:15.502864 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-2354c152-d9b6-43d1-afd9-8b8b45ad6795 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:37:15.619403 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Searching for a device in session 32 and hctl ['2', '*', '*', 0] yield: sda {{(pid=107475) device_name_by_hctl /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:808}} Nov 25 13:37:15.620414 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Connected to sda using {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'target_lun': 0, 'volume_id': 'd41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'auth_method': 'CHAP', 'auth_username': 'jLBUWPmqGR66WohA4V2A', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False} {{(pid=107475) _connect_vol /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:687}} Nov 25 13:37:15.621483 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.183s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:37:15.621911 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] <== connect_volume: return (2182ms) {'type': 'block', 'scsi_wwn': '360014050b8c7aa3e33b4031973795130', 'path': '/dev/sda'} {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:37:15.622465 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:15.650098 np0005535648.novalocal sudo[151765]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda Nov 25 13:37:16.055880 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 1003 /dev/sda" returned: 0 in 0.433s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:16.061560 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] _create_container started, container: tempest-volumesbackupsv39test-backup-container-353768439,backup: 81f1d77a-5ac7-44bb-bf50-ae172ce21df5. {{(pid=107475) _create_container /opt/stack/cinder/cinder/backup/chunkeddriver.py:252}} Nov 25 13:37:16.215642 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] generate_object_name_prefix: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5 {{(pid=107475) _generate_object_name_prefix /opt/stack/cinder/cinder/backup/drivers/swift.py:433}} Nov 25 13:37:16.220066 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] starting backup of volume: d41e9df7-0b74-4c59-84e6-0e3eac93d37f, volume size: 1073741824, object names prefix volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5, availability zone: nova {{(pid=107475) _prepare_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:362}} Nov 25 13:37:16.350370 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:16.488281 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:16.488508 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:16.635157 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00001: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:16.635448 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:16.990720 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:17.153158 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:17.153665 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:17.320048 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00002: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:17.320230 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:17.540661 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:17.703526 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:17.703995 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:17.857130 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00003: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:17.857297 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:18.028992 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:18.170824 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:18.171096 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:18.282820 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00004: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:18.282945 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:18.459939 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:18.569085 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:18.569250 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:18.680037 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00005: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:18.680369 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:19.345999 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:19.483403 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:19.483603 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:19.639648 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00006: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:19.639945 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:20.019106 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:20.141526 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:20.141836 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:20.251327 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00007: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:20.251650 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:20.727944 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:20.897754 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:20.897940 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:21.011721 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00008: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:21.011953 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:21.429680 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:21.525572 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:21.525847 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:21.675832 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00009: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:21.676006 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:21.833163 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:21.966407 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:21.966514 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:22.106007 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00010: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:22.106357 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:22.295419 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:22.432491 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:22.432732 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:22.578458 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00011: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:22.578700 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:22.684606 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:22.812576 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:22.812773 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:22.927728 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00012: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:22.928286 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:23.290909 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:23.419071 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:23.419293 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:23.540972 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00013: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:23.541419 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:23.717964 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:23.841592 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:23.841783 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:23.999274 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00014: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:24.000009 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:24.194898 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:24.330107 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:24.330228 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:24.489801 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00015: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:24.490100 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:24.654901 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:24.821208 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:24.821377 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:24.932922 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00016: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:24.933047 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:25.061345 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:25.175059 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:25.175199 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:25.295835 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00017: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:25.296031 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:25.768558 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:25.910391 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:25.910584 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:26.058095 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00018: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:26.058203 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:26.236096 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:26.372619 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:26.372869 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:26.568003 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00019: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:26.568162 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:26.791525 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:26.886827 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 52428800 bytes of data to 50976 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:26.887103 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:27.092755 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00020: 25e317773f308e446cc84c503a6d1f85 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:27.093015 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:27.616350 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Backing up chunk of data from volume. {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:394}} Nov 25 13:37:27.685075 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Compressed 25165824 bytes of data to 24470 bytes using zlib. {{(pid=107475) _prepare_output_data /opt/stack/cinder/cinder/backup/chunkeddriver.py:433}} Nov 25 13:37:27.685347 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] About to put_object {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:397}} Nov 25 13:37:27.756317 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup MD5 for volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00021: 77377273b0a4b61febdbf7bbf52b9db9 {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:405}} Nov 25 13:37:27.756641 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Calling eventlet.sleep(0) {{(pid=107475) _backup_chunk /opt/stack/cinder/cinder/backup/chunkeddriver.py:412}} Nov 25 13:37:27.763591 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Getting metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:79}} Nov 25 13:37:27.773886 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Completed fetching metadata type 'volume-base-metadata' {{(pid=107475) _save_vol_base_meta /opt/stack/cinder/cinder/backup/driver.py:98}} Nov 25 13:37:27.774154 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Getting metadata type 'volume-metadata' {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:109}} Nov 25 13:37:27.779041 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] No metadata type 'volume-metadata' available {{(pid=107475) _save_vol_meta /opt/stack/cinder/cinder/backup/driver.py:123}} Nov 25 13:37:27.779280 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Getting metadata type 'volume-glance-metadata' {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:132}} Nov 25 13:37:27.784147 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.driver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] No metadata type 'volume-glance-metadata' available {{(pid=107475) _save_vol_glance_meta /opt/stack/cinder/cinder/backup/driver.py:147}} Nov 25 13:37:27.784593 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] _write_sha256file started, container name: tempest-volumesbackupsv39test-backup-container-353768439, sha256file filename: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5_sha256file. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:302}} Nov 25 13:37:27.847592 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] _write_sha256file finished. {{(pid=107475) _write_sha256file /opt/stack/cinder/cinder/backup/chunkeddriver.py:318}} Nov 25 13:37:27.848321 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] _write_metadata started, container name: tempest-volumesbackupsv39test-backup-container-353768439, metadata filename: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5_metadata. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:279}} Nov 25 13:37:27.880566 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] _write_metadata finished. Metadata: b'{\n "backup_description": null,\n "backup_id": "81f1d77a-5ac7-44bb-bf50-ae172ce21df5",\n "backup_name": "tempest-VolumesBackupsV39Test-Backup-477837502",\n "created_at": "2025-11-25 18:37:09+00:00",\n "objects": [\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00001": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 0\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00002": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 52428800\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00003": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 104857600\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00004": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 157286400\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00005": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 209715200\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00006": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 262144000\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00007": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 314572800\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00008": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 367001600\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00009": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 419430400\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00010": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 471859200\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00011": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 524288000\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00012": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 576716800\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00013": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 629145600\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00014": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 681574400\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00015": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 734003200\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00016": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 786432000\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00017": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 838860800\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00018": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 891289600\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00019": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 943718400\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00020": {\n "compression": "zlib",\n "length": 52428800,\n "md5": "25e317773f308e446cc84c503a6d1f85",\n "offset": 996147200\n }\n },\n {\n "volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00021": {\n "compression": "zlib",\n "length": 25165824,\n "md5": "77377273b0a4b61febdbf7bbf52b9db9",\n "offset": 1048576000\n }\n }\n ],\n "parent_id": null,\n "version": "1.0.0",\n "volume_id": "d41e9df7-0b74-4c59-84e6-0e3eac93d37f",\n "volume_meta": "{\\"version\\": 2, \\"volume-base-metadata\\": {\\"id\\": \\"d41e9df7-0b74-4c59-84e6-0e3eac93d37f\\", \\"_name_id\\": null, \\"use_quota\\": true, \\"ec2_id\\": null, \\"user_id\\": \\"e351f755cf754f40925b369d03d6ad5d\\", \\"project_id\\": \\"ef208f8a9de94f50840218015bfcd515\\", \\"snapshot_id\\": null, \\"cluster_name\\": null, \\"host\\": \\"np0005535648.novalocal@lvmdriver-1#lvmdriver-1\\", \\"size\\": 1, \\"availability_zone\\": \\"nova\\", \\"status\\": \\"backing-up\\", \\"attach_status\\": \\"detached\\", \\"migration_status\\": null, \\"scheduled_at\\": \\"2025-11-25T18:37:08.000000\\", \\"launched_at\\": \\"2025-11-25T18:37:09.000000\\", \\"terminated_at\\": null, \\"display_name\\": \\"tempest-VolumesBackupsV39Test-Volume-1092866215\\", \\"display_description\\": null, \\"provider_location\\": \\"38.102.83.58:3260,iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f 0\\", \\"provider_auth\\": \\"CHAP jLBUWPmqGR66WohA4V2A cJMjgFsVYsfV36VP\\", \\"provider_geometry\\": null, \\"provider_id\\": null, \\"volume_type_id\\": \\"42ef2ac8-46e2-415f-ad47-5f278d4fe376\\", \\"source_volid\\": null, \\"encryption_key_id\\": null, \\"consistencygroup_id\\": null, \\"group_id\\": null, \\"bootable\\": false, \\"multiattach\\": false, \\"replication_status\\": null, \\"replication_extended_status\\": null, \\"replication_driver_data\\": null, \\"previous_status\\": \\"available\\", \\"service_uuid\\": \\"e7e65488-c10f-4744-a3b2-5f726bb311ba\\", \\"shared_targets\\": false, \\"deleted_at\\": null, \\"deleted\\": false, \\"created_at\\": \\"2025-11-25T18:37:08.000000\\", \\"updated_at\\": \\"2025-11-25T18:37:11.000000\\"}}"\n}'. {{(pid=107475) _write_metadata /opt/stack/cinder/cinder/backup/chunkeddriver.py:298}} Nov 25 13:37:27.883414 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] backup 81f1d77a-5ac7-44bb-bf50-ae172ce21df5 finished. {{(pid=107475) _finalize_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:465}} Nov 25 13:37:27.950768 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:27.976749 np0005535648.novalocal sudo[151777]: stack : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda Nov 25 13:37:28.364223 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf chown 0 /dev/sda" returned: 0 in 0.413s {{(pid=107475) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.365934 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] ==> disconnect_volume: call "{'args': (, {'target_discovered': False, 'target_portal': '38.102.83.58:3260', 'target_iqn': 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'target_lun': 0, 'volume_id': 'd41e9df7-0b74-4c59-84e6-0e3eac93d37f', 'auth_method': 'CHAP', 'auth_username': 'jLBUWPmqGR66WohA4V2A', 'auth_password': '***', 'encrypted': False, 'qos_specs': None, 'access_mode': 'rw', 'cacheable': False, 'enforce_multipath': False}, {'type': 'block', 'scsi_wwn': '360014050b8c7aa3e33b4031973795130', 'path': '/dev/sda'}), 'kwargs': {'force': True}}" {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:177}} Nov 25 13:37:28.366096 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:68}} Nov 25 13:37:28.366898 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:73}} Nov 25 13:37:28.369318 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.380780 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.012s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.381483 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed622c5f-8bfb-4ffa-a5bc-909da3f4b09e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f\n\tPortal: 38.102.83.58:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS: Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: iSNS: Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: STATIC: Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: Target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: Portal: 38.102.83.58:3260,-1 Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: Iface Name: default Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: FIRMWARE: Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: No targets found. Nov 25 13:37:28.382129 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:37:28.382375 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Regex to get portals from discoverydb: ^SENDTARGETS: Nov 25 13:37:28.382375 np0005535648.novalocal cinder-backup[107475]: .*?^DiscoveryAddress: 38.102.83.58,3260.*? Nov 25 13:37:28.382375 np0005535648.novalocal cinder-backup[107475]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=107475) _get_discoverydb_portals /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:392}} Nov 25 13:37:28.382670 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Getting connected devices for (ips,iqns,luns)=[('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f', 0)] {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:855}} Nov 25 13:37:28.384332 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.394131 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.394341 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[18ac043c-2c95-417f-a1b4-59a456d37875]: (4, ('38.102.83.58:3260,4294967295 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.395191 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.406387 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.011s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.406606 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[835f90d1-0c95-411b-b9ba-7f4ae146ad27]: (4, ('tcp: [32] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f (non-flash)\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.406976 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('-m', 'session'): stdout=tcp: [32] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f (non-flash) Nov 25 13:37:28.406976 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm_bare /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1223}} Nov 25 13:37:28.407238 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsi session list stdout=tcp: [32] 38.102.83.58:3260,1 iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f (non-flash) Nov 25 13:37:28.407238 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsi_session /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1212}} Nov 25 13:37:28.407960 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Resulting device map defaultdict(. at 0x7f9bd9a6ab60>, {('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f'): ({'sda'}, set())}) {{(pid=107475) _get_connection_devices /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:888}} Nov 25 13:37:28.408291 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Removing single pathed devices sda {{(pid=107475) remove_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:381}} Nov 25 13:37:28.409041 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.414092 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] 'multipathd show status' failed. Not Retrying. {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:423}} Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: Exception during request[704ea1c5-5b62-4c60-93aa-640c80095147]: [Errno 2] No such file or directory: 'multipathd' {{(pid=140715) _process_cmd /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:492}} Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: Traceback (most recent call last): Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: ret = func(*f_args, **f_kwargs) Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: return func(*args, **kwargs) Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs) Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: return putils.execute(on_execute=on_execute, Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: obj = subprocess.Popen(cmd, Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: ^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__ Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: self._execute_child(args, executable, preexec_fn, close_fds, Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: raise child_exception_type(errno_num, err_msg, err_filename) Nov 25 13:37:28.415315 np0005535648.novalocal cinder-backup[107475]: FileNotFoundError: [Errno 2] No such file or directory: 'multipathd' Nov 25 13:37:28.416120 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[704ea1c5-5b62-4c60-93aa-640c80095147]: (5, 'builtins.FileNotFoundError', (2, 'No such file or directory'), 'Traceback (most recent call last):\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py", line 489, in _process_cmd\n ret = func(*f_args, **f_kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_privsep/priv_context.py", line 270, in _wrap\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 198, in execute_root\n return custom_execute(*cmd, shell=False, run_as_root=False, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/os_brick/privileged/rootwrap.py", line 146, in custom_execute\n return putils.execute(on_execute=on_execute,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py", line 354, in execute\n obj = subprocess.Popen(cmd,\n ^^^^^^^^^^^^^^^^^^^^^\n File "/usr/lib64/python3.12/subprocess.py", line 1026, in __init__\n self._execute_child(args, executable, preexec_fn, close_fds,\n File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child\n raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: \'multipathd\'\n') {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.416609 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Flushing IO for device /dev/sda {{(pid=107475) flush_device_io /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:441}} Nov 25 13:37:28.417170 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sda {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.424997 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sda" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.425167 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f632fc9-b9ae-43ab-9cf4-ae7fe3b20360]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.425441 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Remove SCSI device /dev/sda with /sys/block/sda/device/delete {{(pid=107475) remove_scsi_device /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:151}} Nov 25 13:37:28.426071 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sda/device/delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.463435 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sda/device/delete" returned: 0 in 0.037s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.463692 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[91f4c823-a9cd-4f11-9c39-39aa7964fce9]: (4, ('1', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.464124 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Checking to see if SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:159}} Nov 25 13:37:28.464357 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.linuxscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] SCSI volumes sda have been removed. {{(pid=107475) wait_for_volumes_removal /usr/local/lib/python3.12/site-packages/os_brick/initiator/linuxscsi.py:169}} Nov 25 13:37:28.464694 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Disconnecting from: [('38.102.83.58:3260', 'iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f')] {{(pid=107475) _disconnect_connection /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1201}} Nov 25 13:37:28.465396 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.startup -v manual {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.473344 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op update -n node.startup -v manual" returned: 0 in 0.008s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.473579 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa9d60a5-aa2e-4c04-bd9c-525c78cce9c1]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.473929 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:28.474559 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --logout {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.498631 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --logout" returned: 0 in 0.023s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.498929 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[9563fa61-8451-448c-aad1-5e631f31f97a]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f, portal: 38.102.83.58,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f, portal: 38.102.83.58,3260] successful.\n', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.499298 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f, portal: 38.102.83.58,3260] Nov 25 13:37:28.499298 np0005535648.novalocal cinder-backup[107475]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f, portal: 38.102.83.58,3260] successful. Nov 25 13:37:28.499298 np0005535648.novalocal cinder-backup[107475]: stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:28.500014 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op delete {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349}} Nov 25 13:37:28.507499 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:volume-d41e9df7-0b74-4c59-84e6-0e3eac93d37f -p 38.102.83.58:3260 --op delete" returned: 0 in 0.007s {{(pid=140715) execute /usr/local/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372}} Nov 25 13:37:28.507722 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe286907-b6a5-4cc8-a110-82f960347af2]: (4, ('', '')) {{(pid=140715) _call_back /usr/local/lib/python3.12/site-packages/oslo_privsep/daemon.py:515}} Nov 25 13:37:28.507987 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=107475) _run_iscsiadm /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/iscsi.py:1067}} Nov 25 13:37:28.508322 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.base [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.142s {{(pid=107475) inner /usr/local/lib/python3.12/site-packages/os_brick/initiator/connectors/base.py:87}} Nov 25 13:37:28.508485 np0005535648.novalocal cinder-backup[107475]: DEBUG os_brick.initiator.connectors.iscsi [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] <== disconnect_volume: return (142ms) None {{(pid=107475) trace_logging_wrapper /usr/local/lib/python3.12/site-packages/os_brick/utils.py:204}} Nov 25 13:37:32.030941 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-95a3e461-33e4-4240-9b08-5a2e7715bb3d tempest-VolumesBackupsV39Test-1458165901 None] Create backup finished. backup: 81f1d77a-5ac7-44bb-bf50-ae172ce21df5. Nov 25 13:37:32.602066 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] Delete backup started, backup: 81f1d77a-5ac7-44bb-bf50-ae172ce21df5. Nov 25 13:37:32.603856 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] Using swift URL https://38.102.83.58:8080/v1/AUTH_ef208f8a9de94f50840218015bfcd515 {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:286}} Nov 25 13:37:32.604179 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.drivers.swift [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] Connect to https://38.102.83.58:8080/v1/AUTH_ in "per_user" mode {{(pid=107475) initialize /opt/stack/cinder/cinder/backup/drivers/swift.py:287}} Nov 25 13:37:32.604509 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] delete started, backup: 81f1d77a-5ac7-44bb-bf50-ae172ce21df5, container: tempest-volumesbackupsv39test-backup-container-353768439, prefix: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:844}} Nov 25 13:37:32.684381 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] generated object list: ['volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00001', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00002', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00003', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00004', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00005', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00006', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00007', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00008', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00009', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00010', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00011', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00012', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00013', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00014', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00015', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00016', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00017', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00018', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00019', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00020', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00021', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5_metadata', 'volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5_sha256file']. {{(pid=107475) _generate_object_names /opt/stack/cinder/cinder/backup/chunkeddriver.py:263}} Nov 25 13:37:32.723801 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00001 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.755704 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00002 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.789746 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00003 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.822477 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00004 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.884190 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00005 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.917746 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00006 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.952625 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00007 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:32.978223 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00008 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.014987 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00009 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.043764 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00010 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.078245 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00011 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.110788 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00012 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.142647 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00013 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.175165 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00014 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.206167 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00015 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.235775 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00016 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.270265 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00017 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.300319 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00018 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.328173 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00019 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.355067 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00020 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.383594 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5-00021 in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.410957 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5_metadata in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.439842 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] deleted object: volume_d41e9df7-0b74-4c59-84e6-0e3eac93d37f/20251125183716/az_nova_backup_81f1d77a-5ac7-44bb-bf50-ae172ce21df5_sha256file in container: tempest-volumesbackupsv39test-backup-container-353768439. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:860}} Nov 25 13:37:33.440306 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.backup.chunkeddriver [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] delete 81f1d77a-5ac7-44bb-bf50-ae172ce21df5 finished. {{(pid=107475) delete_backup /opt/stack/cinder/cinder/backup/chunkeddriver.py:870}} Nov 25 13:37:33.444568 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_db.sqlalchemy.engines [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=107475) _check_effective_sql_mode /usr/local/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:397}} Nov 25 13:37:33.462700 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.quota [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] Created reservations ['0b3b7385-9c4d-4068-a549-b15685f1d9d0', '2929608b-ebe0-4a0d-aff9-7537f48b116b'] {{(pid=107475) reserve /opt/stack/cinder/cinder/quota.py:752}} Nov 25 13:37:33.480514 np0005535648.novalocal cinder-backup[107475]: INFO cinder.backup.manager [None req-66da5d89-9c97-4df2-a04e-b40191d05474 tempest-VolumesBackupsV39Test-1458165901 None] Delete backup finished, backup 81f1d77a-5ac7-44bb-bf50-ae172ce21df5 deleted. Nov 25 13:37:34.623920 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:37:34.624255 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:37:43.481166 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=39,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:38:34.627901 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:38:34.628458 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Nov 25 13:39:25.508388 np0005535648.novalocal cinder-backup[107475]: DEBUG dbcounter [-] [107475] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=107475) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:39:34.634878 np0005535648.novalocal cinder-backup[107475]: DEBUG oslo_service.periodic_task [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Running periodic task BackupManager.publish_service_capabilities {{(pid=107475) run_periodic_tasks /usr/local/lib/python3.12/site-packages/oslo_service/periodic_task.py:210}} Nov 25 13:39:34.635479 np0005535648.novalocal cinder-backup[107475]: DEBUG cinder.manager [None req-d3e628ac-259d-45e0-93a7-c3cabc3b27ea None None] Notifying Schedulers of capabilities ... {{(pid=107475) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}