2026-02-14 14:43:39.299840 | Job console starting...
2026-02-14 14:43:39.311414 | Updating repositories
2026-02-14 14:43:39.352931 | Preparing job workspace
2026-02-14 14:43:46.365908 | Running Ansible setup...
2026-02-14 14:43:50.096539 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/base-minimal/pre.yaml@master]
2026-02-14 14:43:50.739849 | 
2026-02-14 14:43:50.740041 | PLAY [localhost]
2026-02-14 14:43:50.748415 | 
2026-02-14 14:43:50.748479 | TASK [Gathering Facts]
2026-02-14 14:43:51.755547 | localhost | ok
2026-02-14 14:43:51.771787 | 
2026-02-14 14:43:51.771906 | TASK [log-inventory : Ensure Zuul Ansible directory exists]
2026-02-14 14:43:52.131680 | localhost -> localhost | changed
2026-02-14 14:43:52.143207 | 
2026-02-14 14:43:52.143386 | TASK [log-inventory : Copy ansible inventory to logs dir]
2026-02-14 14:43:53.032868 | localhost -> localhost | changed
2026-02-14 14:43:53.041450 | 
2026-02-14 14:43:53.041516 | TASK [Setup log path fact]
2026-02-14 14:43:53.058872 | localhost | ok
2026-02-14 14:43:53.069810 | 
2026-02-14 14:43:53.069869 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-02-14 14:43:53.107628 | localhost | ok
2026-02-14 14:43:53.114891 | 
2026-02-14 14:43:53.114960 | TASK [emit-job-header : Print job information]
2026-02-14 14:43:53.163832 | # Job Information
2026-02-14 14:43:53.163982 | Ansible Version: 2.15.12
2026-02-14 14:43:53.164036 | Job: openstack-baremetal-operator-hotstack-bm
2026-02-14 14:43:53.164071 | Pipeline: github-check
2026-02-14 14:43:53.164097 | Executor: ze01.softwarefactory-project.io
2026-02-14 14:43:53.164120 | Triggered by: https://github.com/openstack-k8s-operators/openstack-baremetal-operator/pull/376
2026-02-14 14:43:53.164145 | Log URL (when completed): https://logserver.rdoproject.org/e16/rdoproject.org/e16e435992bd479db8414c1353b1e0de/
2026-02-14 14:43:53.164168 | Event ID: 0b7b3000-09b1-11f1-86b3-9e67492861e1
2026-02-14 14:43:53.169616 | 
2026-02-14 14:43:53.169703 | LOOP [emit-job-header : Print node information]
2026-02-14 14:43:53.271462 | localhost | ok:
2026-02-14 14:43:53.271672 | localhost | # Node Information
2026-02-14 14:43:53.271700 | localhost | Inventory Hostname: hotstack
2026-02-14 14:43:53.271726 | localhost | Hostname: np0005619768
2026-02-14 14:43:53.271745 | localhost | Username: zuul
2026-02-14 14:43:53.271765 | localhost | Distro: CentOS 9
2026-02-14 14:43:53.271782 | localhost | Provider: vexxhost-nodepool-tripleo
2026-02-14 14:43:53.271799 | localhost | Region: RegionOne
2026-02-14 14:43:53.271815 | localhost | Label: cloud-centos-9-stream-tripleo-vexxhost
2026-02-14 14:43:53.271831 | localhost | Product Name: OpenStack Nova
2026-02-14 14:43:53.271846 | localhost | Interface IP: 38.102.83.115
2026-02-14 14:43:53.314429 | 
2026-02-14 14:43:53.314583 | PLAY [all]
2026-02-14 14:43:53.328307 | 
2026-02-14 14:43:53.328432 | TASK [Gather network facts]
2026-02-14 14:43:53.769597 | hotstack | ok
2026-02-14 14:43:53.784507 | 
2026-02-14 14:43:53.784566 | TASK [include_role : start-zuul-console]
2026-02-14 14:43:53.812989 | hotstack | ok
2026-02-14 14:43:53.825046 | 
2026-02-14 14:43:53.825110 | TASK [start-zuul-console : Start zuul_console daemon.]
2026-02-14 14:43:54.248868 | hotstack | ok
2026-02-14 14:43:54.271916 | 
2026-02-14 14:43:54.272214 | TASK [include_role : add-build-sshkey]
2026-02-14 14:43:54.307599 | hotstack | ok
2026-02-14 14:43:54.328982 | 
2026-02-14 14:43:54.329183 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build]
2026-02-14 14:43:54.570866 | hotstack -> localhost | ok
2026-02-14 14:43:54.583924 | 
2026-02-14 14:43:54.584131 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID]
2026-02-14 14:43:54.622986 | hotstack | ok
2026-02-14 14:43:54.651744 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml
2026-02-14 14:43:54.660488 | 
2026-02-14 14:43:54.660581 | TASK [add-build-sshkey : Create Temp SSH key]
2026-02-14 14:43:55.293543 | hotstack -> localhost | Generating public/private rsa key pair.
2026-02-14 14:43:55.293856 | hotstack -> localhost | Your identification has been saved in /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/e16e435992bd479db8414c1353b1e0de_id_rsa.
2026-02-14 14:43:55.293937 | hotstack -> localhost | Your public key has been saved in /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/e16e435992bd479db8414c1353b1e0de_id_rsa.pub.
2026-02-14 14:43:55.293969 | hotstack -> localhost | The key fingerprint is:
2026-02-14 14:43:55.293995 | hotstack -> localhost | SHA256:/NgTOzCBvlJgGyloxlF6vu3pwJx/yLoxb4k8MmLyEH4 zuul-build-sshkey
2026-02-14 14:43:55.294108 | hotstack -> localhost | The key's randomart image is:
2026-02-14 14:43:55.294174 | hotstack -> localhost | +---[RSA 3072]----+
2026-02-14 14:43:55.294265 | hotstack -> localhost | | ...             |
2026-02-14 14:43:55.294321 | hotstack -> localhost | |..o  . .         |
2026-02-14 14:43:55.294347 | hotstack -> localhost | |.=..= . .        |
2026-02-14 14:43:55.294372 | hotstack -> localhost | |o oo = . .       |
2026-02-14 14:43:55.294396 | hotstack -> localhost | |.  .. o S .      |
2026-02-14 14:43:55.294420 | hotstack -> localhost | |..o +. . * o     |
2026-02-14 14:43:55.294477 | hotstack -> localhost | |...E+.+ . *      |
2026-02-14 14:43:55.294545 | hotstack -> localhost | |o=.+B*..   o     |
2026-02-14 14:43:55.294613 | hotstack -> localhost | |+.+o**.          |
2026-02-14 14:43:55.294639 | hotstack -> localhost | +----[SHA256]-----+
2026-02-14 14:43:55.294703 | hotstack -> localhost | ok: Runtime: 0:00:00.085439
2026-02-14 14:43:55.307785 | 
2026-02-14 14:43:55.307936 | TASK [add-build-sshkey : Remote setup ssh keys (linux)]
2026-02-14 14:43:55.341100 | hotstack | ok
2026-02-14 14:43:55.353707 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml
2026-02-14 14:43:55.363367 | 
2026-02-14 14:43:55.363454 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey]
2026-02-14 14:43:55.389070 | hotstack | skipping: Conditional result was False
2026-02-14 14:43:55.395153 | 
2026-02-14 14:43:55.395219 | TASK [add-build-sshkey : Enable access via build key on all nodes]
2026-02-14 14:43:55.862240 | hotstack | changed
2026-02-14 14:43:55.871476 | 
2026-02-14 14:43:55.871563 | TASK [add-build-sshkey : Make sure user has a .ssh]
2026-02-14 14:43:56.156428 | hotstack | ok
2026-02-14 14:43:56.178630 | 
2026-02-14 14:43:56.178785 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes]
2026-02-14 14:43:56.972478 | hotstack | changed
2026-02-14 14:43:56.979428 | 
2026-02-14 14:43:56.979508 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes]
2026-02-14 14:43:57.814153 | hotstack | changed
2026-02-14 14:43:57.840587 | 
2026-02-14 14:43:57.840734 | TASK [add-build-sshkey : Remote setup ssh keys (windows)]
2026-02-14 14:43:57.867307 | hotstack | skipping: Conditional result was False
2026-02-14 14:43:57.883467 | 
2026-02-14 14:43:57.883558 | TASK [remove-zuul-sshkey : Remove master key from local agent]
2026-02-14 14:43:58.304234 | hotstack -> localhost | changed
2026-02-14 14:43:58.317285 | 
2026-02-14 14:43:58.317374 | TASK [add-build-sshkey : Add back temp key]
2026-02-14 14:43:58.680581 | hotstack -> localhost | Identity added: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/e16e435992bd479db8414c1353b1e0de_id_rsa (zuul-build-sshkey)
2026-02-14 14:43:58.680971 | hotstack -> localhost | ok: Runtime: 0:00:00.014992
2026-02-14 14:43:58.702150 | 
2026-02-14 14:43:58.702282 | TASK [add-build-sshkey : Verify we can still SSH to all nodes]
2026-02-14 14:43:59.057370 | hotstack | ok
2026-02-14 14:43:59.067745 | 
2026-02-14 14:43:59.067872 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)]
2026-02-14 14:43:59.104501 | hotstack | skipping: Conditional result was False
2026-02-14 14:43:59.126838 | 
2026-02-14 14:43:59.127193 | TASK [include_role : validate-host]
2026-02-14 14:43:59.170392 | hotstack | ok
2026-02-14 14:43:59.197602 | 
2026-02-14 14:43:59.197672 | TASK [validate-host : Define zuul_info_dir fact]
2026-02-14 14:43:59.236364 | hotstack | ok
2026-02-14 14:43:59.240806 | 
2026-02-14 14:43:59.240863 | TASK [validate-host : Ensure Zuul Ansible directory exists]
2026-02-14 14:43:59.477624 | hotstack -> localhost | ok
2026-02-14 14:43:59.485057 | 
2026-02-14 14:43:59.485124 | TASK [validate-host : Collect information about the host]
2026-02-14 14:44:00.244126 | hotstack | ok
2026-02-14 14:44:00.262749 | 
2026-02-14 14:44:00.262876 | TASK [validate-host : Sanitize hostname]
2026-02-14 14:44:00.321502 | hotstack | ok
2026-02-14 14:44:00.332392 | 
2026-02-14 14:44:00.332572 | TASK [validate-host : Write out all ansible variables/facts known for each host]
2026-02-14 14:44:00.812260 | hotstack -> localhost | changed
2026-02-14 14:44:00.823780 | 
2026-02-14 14:44:00.823939 | TASK [validate-host : Collect information about zuul worker]
2026-02-14 14:44:01.229878 | hotstack | ok
2026-02-14 14:44:01.240222 | 
2026-02-14 14:44:01.240390 | TASK [validate-host : Write out all zuul information for each host]
2026-02-14 14:44:01.747971 | hotstack -> localhost | changed
2026-02-14 14:44:01.767997 | 
2026-02-14 14:44:01.768199 | TASK [include_role : prepare-workspace-openshift]
2026-02-14 14:44:01.785135 | hotstack | skipping: Conditional result was False
2026-02-14 14:44:01.796676 | 
2026-02-14 14:44:01.796796 | TASK [include_role : remove-zuul-sshkey]
2026-02-14 14:44:01.820054 | hotstack | skipping: Conditional result was False
2026-02-14 14:44:01.825629 | 
2026-02-14 14:44:01.825690 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them]
2026-02-14 14:44:02.138670 | hotstack | ok: "logs"
2026-02-14 14:44:02.139419 | hotstack | ok: All items complete
2026-02-14 14:44:02.139508 | 
2026-02-14 14:44:02.367806 | hotstack | ok: "artifacts"
2026-02-14 14:44:02.578168 | hotstack | ok: "docs"
2026-02-14 14:44:02.597675 | 
2026-02-14 14:44:02.597861 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist]
2026-02-14 14:44:02.987941 | hotstack | changed: "logs"
2026-02-14 14:44:03.224927 | hotstack | changed: "artifacts"
2026-02-14 14:44:03.478489 | hotstack | changed: "docs"
2026-02-14 14:44:03.529329 | 
2026-02-14 14:44:03.529498 | PLAY RECAP
2026-02-14 14:44:03.529582 | hotstack | ok: 22 changed: 9 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0
2026-02-14 14:44:03.529638 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0
2026-02-14 14:44:03.529678 | 
2026-02-14 14:44:03.650519 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/base-minimal/pre.yaml@master]
2026-02-14 14:44:03.657838 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/hotstack/pre_stage1.yaml@master]
2026-02-14 14:44:04.286856 | 
2026-02-14 14:44:04.286966 | PLAY [Create necessary files to deploy hotstack scenario]
2026-02-14 14:44:04.307682 | 
2026-02-14 14:44:04.307758 | TASK [prepare-workspace : Start zuul_console daemon.]
2026-02-14 14:44:04.699641 | hotstack | ok
2026-02-14 14:44:04.712818 | 
2026-02-14 14:44:04.712987 | TASK [prepare-workspace : Synchronize src repos to workspace directory.]
2026-02-14 14:44:07.741920 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:07.763319 | 
2026-02-14 14:44:07.763493 | LOOP [add-authorized-keys : Enable access via build key on all nodes]
2026-02-14 14:44:08.239706 | hotstack | changed:
2026-02-14 14:44:08.240044 | hotstack | {
2026-02-14 14:44:08.240116 | hotstack |   "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCINPOs6XsfMVQMqjmiduT3Bxhogy7MbAFOWlR+9LlKkTW1kV6glsCXY5svtggd2zRtc40UWalHaj9jC+yEkZdI35h2PkK+Sww52u4nczqgNpIFNt7Dm2xtYKbTolAV9jSeNkRlGMCq9EBzQlwiV0HkHx1CdNR/zzXbyNZLmOybFkXqR150H01kUhdv0v6AMWTDZjMEWO/PaWCOgzZhH9ztXjy+rwa7pdaI4+0Pb9Kib+d8/IrgNsLB/sCazqD+9/COI6AHqzTDS1aU9/gMQrUN1ronzMEQhzzOuoI8WeLqU++j+0dT2UWVPlQqoln+RYu7dUn/DSZDrsYenudT5+P"
2026-02-14 14:44:08.240163 | hotstack | }
2026-02-14 14:44:08.264389 | 
2026-02-14 14:44:08.264586 | TASK [Assert cloud name and cloud secret]
2026-02-14 14:44:08.345519 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:08.357239 | 
2026-02-14 14:44:08.357403 | TASK [Ensure openstack release repo is installed]
2026-02-14 14:44:12.557666 | hotstack | changed
2026-02-14 14:44:12.562678 | 
2026-02-14 14:44:12.562741 | TASK [Install openstack client packages]
2026-02-14 14:44:27.028646 | hotstack | changed
2026-02-14 14:44:27.036457 | 
2026-02-14 14:44:27.036558 | TASK [Ensure HotStack work dir]
2026-02-14 14:44:27.473630 | hotstack | changed
2026-02-14 14:44:27.484845 | 
2026-02-14 14:44:27.484975 | LOOP [Ensure etc directories]
2026-02-14 14:44:27.811368 | hotstack | changed: "/etc/openstack"
2026-02-14 14:44:27.812351 | hotstack | changed: All items complete
2026-02-14 14:44:27.812445 | 
2026-02-14 14:44:28.061282 | hotstack | changed: "/etc/hotstack"
2026-02-14 14:44:28.082766 | 
2026-02-14 14:44:28.082962 | TASK [Ensure .ssh dir exists]
2026-02-14 14:44:28.327150 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:28.338747 | 
2026-02-14 14:44:28.338873 | TASK [Ensure ssh keypair exists]
2026-02-14 14:44:38.900892 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:38.912658 | 
2026-02-14 14:44:38.912787 | TASK [Generate CA cert file]
2026-02-14 14:44:38.969860 | hotstack | skipping: Conditional result was False
2026-02-14 14:44:38.982375 | 
2026-02-14 14:44:38.982572 | TASK [Update CA certs]
2026-02-14 14:44:39.532002 | hotstack | skipping: Conditional result was False
2026-02-14 14:44:39.538157 | 
2026-02-14 14:44:39.538222 | TASK [Set _auth_type fact]
2026-02-14 14:44:39.587303 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:39.591742 | 
2026-02-14 14:44:39.591800 | TASK [Write clouds.yaml - user/password]
2026-02-14 14:44:40.710188 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:40.721978 | 
2026-02-14 14:44:40.722139 | TASK [Create application credential for sushy-emulator]
2026-02-14 14:44:43.270783 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:43.283679 | 
2026-02-14 14:44:43.283855 | TASK [Read application credential from file]
2026-02-14 14:44:43.669260 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:43.680727 | 
2026-02-14 14:44:43.680857 | TASK [Delete temporary application credential file]
2026-02-14 14:44:43.913777 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:43.925007 | 
2026-02-14 14:44:43.925170 | TASK [Assert application credential id and secret]
2026-02-14 14:44:44.047006 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:44.093945 | 
2026-02-14 14:44:44.094110 | TASK [Write new clouds.yaml with application credentials]
2026-02-14 14:44:44.866577 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:44.877649 | 
2026-02-14 14:44:44.877778 | TASK [Write hotstack cloud-secret.yaml]
2026-02-14 14:44:45.656570 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:45.664084 | 
2026-02-14 14:44:45.664165 | TASK [Write clouds.yaml with application credentials]
2026-02-14 14:44:45.696302 | 
2026-02-14 14:44:45.696457 | TASK [Write hotstack cloud-secret.yaml]
2026-02-14 14:44:45.748340 | 
2026-02-14 14:44:45.748586 | PLAY RECAP
2026-02-14 14:44:45.748680 | hotstack | ok: 18 changed: 11 unreachable: 0 failed: 0 skipped: 4 rescued: 0 ignored: 0
2026-02-14 14:44:45.748724 | 
2026-02-14 14:44:45.857437 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/hotstack/pre_stage1.yaml@master]
2026-02-14 14:44:45.864569 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/hotstack/pre_stage2.yaml@master]
2026-02-14 14:44:46.440034 | 
2026-02-14 14:44:46.440195 | PLAY [Create hotstack scenario]
2026-02-14 14:44:46.461369 | 
2026-02-14 14:44:46.461441 | TASK [Assert scenario is defined]
2026-02-14 14:44:46.550443 | hotstack | ok: All assertions passed
2026-02-14 14:44:46.555603 | 
2026-02-14 14:44:46.555670 | TASK [Fetch bootstrap vars from remote to zuul executor]
2026-02-14 14:44:47.011052 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:47.022049 | 
2026-02-14 14:44:47.022184 | TASK [Load bootstrap vars with template resolution]
2026-02-14 14:44:47.071829 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:47.081994 | 
2026-02-14 14:44:47.082176 | TASK [Fetch automation vars from remote to zuul executor]
2026-02-14 14:44:47.940473 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:47.950389 | 
2026-02-14 14:44:47.950509 | TASK [Load automation vars with template resolution]
2026-02-14 14:44:48.007834 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:48.019707 | 
2026-02-14 14:44:48.019841 | TASK [Slurp zuul-build-sshkey from file]
2026-02-14 14:44:48.382239 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:48.392516 | 
2026-02-14 14:44:48.392644 | TASK [Set up infrastructure (Heat stack)]
2026-02-14 14:44:48.419314 | hotstack | Output suppressed because no_log was given
2026-02-14 14:44:48.476662 | 
2026-02-14 14:44:48.476768 | TASK [dataplane_ssh_keys : Ensure Dataplane ssh keypair exists]
2026-02-14 14:44:49.032146 | hotstack | changed
2026-02-14 14:44:49.043766 | 
2026-02-14 14:44:49.043898 | TASK [dataplane_ssh_keys : Ensure Nova migration ssh keypair exists]
2026-02-14 14:44:49.418042 | hotstack | changed
2026-02-14 14:44:49.431194 | 
2026-02-14 14:44:49.431321 | TASK [dataplane_ssh_keys : Set SSH public key facts]
2026-02-14 14:44:49.469967 | hotstack | ok
2026-02-14 14:44:49.482665 | 
2026-02-14 14:44:49.482796 | TASK [dataplane_ssh_keys : Store SSH public key facts in file]
2026-02-14 14:44:50.425341 | hotstack | changed
2026-02-14 14:44:50.447354 | 
2026-02-14 14:44:50.447532 | TASK [heat_stack : Assert config is defined]
2026-02-14 14:44:50.558797 | hotstack | ok: All assertions passed
2026-02-14 14:44:50.569804 | 
2026-02-14 14:44:50.569933 | TASK [heat_stack : Get latest snapset and update stack parameters]
2026-02-14 14:44:50.617708 | hotstack | skipping: Conditional result was False
2026-02-14 14:44:50.630992 | 
2026-02-14 14:44:50.631179 | TASK [heat_stack : Debug latest snapset information]
2026-02-14 14:44:50.678538 | hotstack | skipping: Conditional result was False
2026-02-14 14:44:50.691654 | 
2026-02-14 14:44:50.691825 | LOOP [heat_stack : Create tar archives and compress]
2026-02-14 14:44:50.721126 | 
2026-02-14 14:44:50.721401 | TASK [heat_stack : Create stack]
2026-02-14 14:46:02.044456 | hotstack | changed
2026-02-14 14:46:02.060791 | 
2026-02-14 14:46:02.060937 | TASK [heat_stack : Wait for stack to complete]
2026-02-14 14:46:04.185432 | hotstack | ok
2026-02-14 14:46:04.197864 | 
2026-02-14 14:46:04.198097 | TASK [heat_stack : Stop if stack not complete]
2026-02-14 14:46:04.225175 | hotstack | skipping: Conditional result was False
2026-02-14 14:46:04.238448 | 
2026-02-14 14:46:04.238650 | TASK [heat_stack : Get stack outputs]
2026-02-14 14:46:06.133534 | hotstack | changed
2026-02-14 14:46:06.147971 | 
2026-02-14 14:46:06.148134 | TASK [heat_stack : Set stack outputs facts]
2026-02-14 14:46:06.196198 | hotstack | ok
2026-02-14 14:46:06.210666 | 
2026-02-14 14:46:06.210799 | TASK [heat_stack : Store stack outputs in file]
2026-02-14 14:46:07.013778 | hotstack | changed
2026-02-14 14:46:07.034932 | 
2026-02-14 14:46:07.035092 | TASK [Fetch stack outputs from remote to zuul executor]
2026-02-14 14:46:07.391352 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:07.402828 | 
2026-02-14 14:46:07.402994 | TASK [Load stack output vars with template resolution]
2026-02-14 14:46:07.453391 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:07.464315 | 
2026-02-14 14:46:07.464440 | TASK [Fetch cloud secrets from remote to zuul executor]
2026-02-14 14:46:07.810259 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:07.820621 | 
2026-02-14 14:46:07.820747 | TASK [Load hotstack cloud secret vars with template resolution]
2026-02-14 14:46:07.869620 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:07.880737 | 
2026-02-14 14:46:07.880947 | TASK [Add Hotstack controller-0 to ansible inventory]
2026-02-14 14:46:07.930504 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:07.942782 | 
2026-02-14 14:46:07.942918 | TASK [Wait for Hotstack controller-0 to be ready]
2026-02-14 14:46:14.203437 | hotstack -> 38.102.83.12 | Output suppressed because no_log was given
2026-02-14 14:46:14.211305 | 
2026-02-14 14:46:14.211390 | TASK [Start Zuul console on Hotstack controller-0]
2026-02-14 14:46:14.241347 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:14.277892 | 
2026-02-14 14:46:14.277978 | TASK [start-zuul-console : Start zuul_console daemon.]
2026-02-14 14:46:15.131871 | hotstack -> 38.102.83.12 | ok
2026-02-14 14:46:15.153472 | 
2026-02-14 14:46:15.153598 | TASK [Prepare the controller node]
2026-02-14 14:46:15.184488 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:15.276491 | 
2026-02-14 14:46:15.276613 | TASK [controller : Assert config is defined]
2026-02-14 14:46:15.356260 | hotstack | ok: All assertions passed
2026-02-14 14:46:15.361151 | 
2026-02-14 14:46:15.361211 | TASK [controller : Add controller-0 to the Ansible inventory]
2026-02-14 14:46:15.390211 | hotstack | ok
2026-02-14 14:46:15.395245 | 
2026-02-14 14:46:15.395305 | TASK [controller : Read Dataplane SSH private key content]
2026-02-14 14:46:15.636617 | hotstack | ok
2026-02-14 14:46:15.647655 | 
2026-02-14 14:46:15.647788 | TASK [controller : Read Nova migration SSH private key content]
2026-02-14 14:46:15.900210 | hotstack | ok
2026-02-14 14:46:15.912145 | 
2026-02-14 14:46:15.912347 | TASK [controller : Wait for controller-0 to be ready]
2026-02-14 14:46:16.714573 | hotstack -> 38.102.83.12 | ok
2026-02-14 14:46:16.727550 | 
2026-02-14 14:46:16.727709 | LOOP [controller : Ensure directories exists]
2026-02-14 14:46:17.515141 | hotstack -> 38.102.83.12 | changed: "/home/zuul/data"
2026-02-14 14:46:17.515381 | 
2026-02-14 14:46:18.147273 | hotstack -> 38.102.83.12 | changed: "/home/zuul/bin"
2026-02-14 14:46:18.166461 | 
2026-02-14 14:46:18.166646 | LOOP [controller : Add common functions to the bin_dir]
2026-02-14 14:46:19.569056 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-common-funcs.sh"
2026-02-14 14:46:19.592933 | 
2026-02-14 14:46:19.593193 | LOOP [controller : Add bin utilities to the bin_dir]
2026-02-14 14:46:20.999996 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-approve-install-plan"
2026-02-14 14:46:21.000446 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 14:46:21.000503 | 
2026-02-14 14:46:22.300227 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-collect-tempest-logs"
2026-02-14 14:46:23.622525 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-leader-election-tune"
2026-02-14 14:46:25.004455 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-nova-discover-hosts"
2026-02-14 14:46:26.377091 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-snapset"
2026-02-14 14:46:27.399181 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-openstack-version-patch"
2026-02-14 14:46:27.962285 | hotstack -> 38.102.83.12 | changed: "bin/hotstack-wait-for-bmh"
2026-02-14 14:46:27.983418 | 
2026-02-14 14:46:27.983612 | TASK [controller : Write ansible inventory to file on controller-0]
2026-02-14 14:46:28.061417 | hotstack | skipping: Conditional result was False
2026-02-14 14:46:28.074156 | 
2026-02-14 14:46:28.074327 | TASK [controller : Copy Dataplane ssh private key to controller]
2026-02-14 14:46:29.180779 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:29.190867 | 
2026-02-14 14:46:29.190997 | TASK [controller : Copy Dataplane ssh public key to controller]
2026-02-14 14:46:29.828283 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:29.843090 | 
2026-02-14 14:46:29.843248 | TASK [controller : Copy Nova migrate ssh private key to controller]
2026-02-14 14:46:30.575998 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:30.596884 | 
2026-02-14 14:46:30.597136 | TASK [controller : Copy Nova migrate ssh public key to controller]
2026-02-14 14:46:31.321974 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:31.336345 | 
2026-02-14 14:46:31.336487 | TASK [controller : Copy dataplane authorized keys to controller]
2026-02-14 14:46:32.077827 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:32.090952 | 
2026-02-14 14:46:32.091184 | TASK [controller : Ensure cloud config directory exists]
2026-02-14 14:46:32.404684 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:32.419065 | 
2026-02-14 14:46:32.419240 | TASK [controller : Template the clouds.yaml]
2026-02-14 14:46:33.175502 | hotstack -> 38.102.83.12 | Output suppressed because no_log was given
2026-02-14 14:46:33.184309 | 
2026-02-14 14:46:33.184404 | TASK [controller : Write CA certificate to file]
2026-02-14 14:46:33.903596 | hotstack -> 38.102.83.12 | Output suppressed because no_log was given
2026-02-14 14:46:33.924972 | 
2026-02-14 14:46:33.925169 | TASK [controller : Write Ironic nodes YAML]
2026-02-14 14:46:34.675289 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:34.701556 | 
2026-02-14 14:46:34.701685 | TASK [Install Openshift]
2026-02-14 14:46:34.730107 | hotstack | Output suppressed because no_log was given
2026-02-14 14:46:34.780074 | 
2026-02-14 14:46:34.780198 | TASK [ocp_agent_installer : Assert config is defined]
2026-02-14 14:46:34.864703 | hotstack -> 38.102.83.12 | ok: All assertions passed
2026-02-14 14:46:34.871915 | 
2026-02-14 14:46:34.871995 | TASK [ocp_agent_installer : Install using agent installer]
2026-02-14 14:46:34.922294 | hotstack | ok
2026-02-14 14:46:34.938135 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/_install.yml
2026-02-14 14:46:34.946744 | 
2026-02-14 14:46:34.946825 | TASK [ocp_agent_installer : Assert config is defined]
2026-02-14 14:46:35.011798 | hotstack -> 38.102.83.12 | ok: All assertions passed
2026-02-14 14:46:35.018242 | 
2026-02-14 14:46:35.018314 | LOOP [ocp_agent_installer : Ensure directory exists]
2026-02-14 14:46:35.367631 | hotstack -> 38.102.83.12 | ok: "/home/zuul/bin"
2026-02-14 14:46:35.650760 | hotstack -> 38.102.83.12 | changed: "/home/zuul/.kube"
2026-02-14 14:46:35.921562 | hotstack -> 38.102.83.12 | changed: "/home/zuul/ocp-cluster"
2026-02-14 14:46:36.199274 | hotstack -> 38.102.83.12 | changed: "/home/zuul/ocp-cluster/openshift"
2026-02-14 14:46:36.462550 | hotstack -> 38.102.83.12 | changed: "/home/zuul/agent-installer"
2026-02-14 14:46:36.805491 | hotstack -> 38.102.83.12 | changed: "/home/zuul/cluster-custom-config/"
2026-02-14 14:46:36.998625 | hotstack -> 38.102.83.12 | changed: "/home/zuul/cluster-custom-config//butane"
2026-02-14 14:46:37.278312 | hotstack -> 38.102.83.12 | changed: "/home/zuul/cluster-custom-config//machine-configs"
2026-02-14 14:46:37.605464 | hotstack -> 38.102.83.12 | changed: "/home/zuul/cluster-custom-config//config-assets"
2026-02-14 14:46:37.627835 | 
2026-02-14 14:46:37.628009 | TASK [ocp_agent_installer : Install package requirements for agent installer]
2026-02-14 14:46:40.611817 | hotstack -> 38.102.83.12 | ok: Nothing to do
2026-02-14 14:46:40.625984 | 
2026-02-14 14:46:40.626160 | TASK [ocp_agent_installer : Run tasks/install_client.yml]
2026-02-14 14:46:40.664755 | hotstack | ok
2026-02-14 14:46:40.680465 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/install_client.yml
2026-02-14 14:46:40.701821 | 
2026-02-14 14:46:40.701911 | TASK [ocp_agent_installer : Download the client]
2026-02-14 14:46:42.651775 | hotstack -> 38.102.83.12 | changed: OK (73365319 bytes)
2026-02-14 14:46:42.676785 | 
2026-02-14 14:46:42.676883 | TASK [ocp_agent_installer : Extract client to /home/zuul/bin]
2026-02-14 14:46:46.993122 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:47.000573 | 
2026-02-14 14:46:47.000651 | TASK [ocp_agent_installer : Configure bash completion]
2026-02-14 14:46:48.026354 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:46:48.039265 | 
2026-02-14 14:46:48.039432 | TASK [ocp_agent_installer : Run tasks/install_installer.yml]
2026-02-14 14:46:48.077498 | hotstack | ok
2026-02-14 14:46:48.091973 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/install_installer.yml
2026-02-14 14:46:48.113815 | 
2026-02-14 14:46:48.113937 | TASK [ocp_agent_installer : Download the installer]
2026-02-14 14:46:56.699468 | hotstack -> 38.102.83.12 | changed: OK (437717358 bytes)
2026-02-14 14:46:56.712148 | 
2026-02-14 14:46:56.712280 | TASK [ocp_agent_installer : Extract installer to /home/zuul/bin]
2026-02-14 14:47:10.785234 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:10.798792 | 
2026-02-14 14:47:10.798935 | TASK [ocp_agent_installer : Write OCP install-config.yaml]
2026-02-14 14:47:11.581106 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:11.594565 | 
2026-02-14 14:47:11.594707 | TASK [ocp_agent_installer : Replace pull secret in install-config.yaml]
2026-02-14 14:47:12.070525 | hotstack -> 38.102.83.12 | Output suppressed because no_log was given
2026-02-14 14:47:12.083101 | 
2026-02-14 14:47:12.083237 | TASK [ocp_agent_installer : Write OCP agent-config.yaml]
2026-02-14 14:47:12.732660 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:12.747639 | 
2026-02-14 14:47:12.747774 | TASK [ocp_agent_installer : Include tasks to generate machine configs]
2026-02-14 14:47:12.785850 | hotstack | ok
2026-02-14 14:47:12.806175 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/machine_configs.yml
2026-02-14 14:47:12.828485 | 
2026-02-14 14:47:12.828577 | LOOP [ocp_agent_installer : Template butane config for net.ifnames]
2026-02-14 14:47:13.530114 | hotstack -> 38.102.83.12 | changed: "master"
2026-02-14 14:47:13.530602 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 14:47:13.530663 | 
2026-02-14 14:47:13.554010 | 
2026-02-14 14:47:13.554330 | LOOP [ocp_agent_installer : Generate MachineConfig for iscsi]
2026-02-14 14:47:13.980705 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:13.995564 | 
2026-02-14 14:47:13.995749 | LOOP [ocp_agent_installer : Template butane config for iscsi]
2026-02-14 14:47:14.765907 | hotstack -> 38.102.83.12 | changed: "master"
2026-02-14 14:47:14.766342 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 14:47:14.766403 | 
2026-02-14 14:47:14.790317 | 
2026-02-14 14:47:14.790477 | LOOP [ocp_agent_installer : Generate MachineConfig for iscsi]
2026-02-14 14:47:15.203591 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:15.218670 | 
2026-02-14 14:47:15.218817 | LOOP [ocp_agent_installer : Template butane config for multipath]
2026-02-14 14:47:16.005567 | hotstack -> 38.102.83.12 | changed: "master"
2026-02-14 14:47:16.005776 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 14:47:16.005804 | 
2026-02-14 14:47:16.020805 | 
2026-02-14 14:47:16.020899 | LOOP [ocp_agent_installer : Generate MachineConfig for multipath]
2026-02-14 14:47:16.341760 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:16.358579 | 
2026-02-14 14:47:16.358725 | LOOP [ocp_agent_installer : Template butane config for LVM cinder-volumes]
2026-02-14 14:47:17.085420 | hotstack -> 38.102.83.12 | changed: "master"
2026-02-14 14:47:17.085688 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 14:47:17.085724 | 
2026-02-14 14:47:17.099588 | 
2026-02-14 14:47:17.099784 | LOOP [ocp_agent_installer : Generate MachineConfig for multipath]
2026-02-14 14:47:17.542461 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:17.603365 | 
2026-02-14 14:47:17.603515 | TASK [ocp_agent_installer : Include tasks to generate config assets]
2026-02-14 14:47:17.636665 | hotstack | ok
2026-02-14 14:47:17.654197 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/config_assets.yml
2026-02-14 14:47:17.679845 | 
2026-02-14 14:47:17.679937 | TASK [ocp_agent_installer : Template ovn-k8s customization]
2026-02-14 14:47:18.407479 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:18.423105 | 
2026-02-14 14:47:18.423244 | TASK [ocp_agent_installer : Copy Etcd customization]
2026-02-14 14:47:19.144819 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:19.158005 | 
2026-02-14 14:47:19.158164 | TASK [ocp_agent_installer : Template ImageContentSourcePolicy customization]
2026-02-14 14:47:19.227100 | hotstack | skipping: Conditional result was False
2026-02-14 14:47:19.240163 | 
2026-02-14 14:47:19.240300 | TASK [ocp_agent_installer : Initialize _ocp_additional_trusted_ca_map fact]
2026-02-14 14:47:19.298720 | hotstack | skipping: Conditional result was False
2026-02-14 14:47:19.311438 | 
2026-02-14 14:47:19.311574 | LOOP [ocp_agent_installer : Append to _ocp_additional_trusted_ca_map fact]
2026-02-14 14:47:19.362302 | hotstack | skipping: Conditional result was False
2026-02-14 14:47:19.362769 | hotstack | changed: All items complete
2026-02-14 14:47:19.362854 | 
2026-02-14 14:47:19.382121 | 
2026-02-14 14:47:19.382285 | TASK [ocp_agent_installer : Template additional CA config map]
2026-02-14 14:47:19.430423 | hotstack | skipping: Conditional result was False
2026-02-14 14:47:19.443000 | 
2026-02-14 14:47:19.443182 | TASK [ocp_agent_installer : Template Image config (additionalTrustedCA and/or insecureRegistries)]
2026-02-14 14:47:19.495816 | hotstack | skipping: Conditional result was False
2026-02-14 14:47:19.507623 | 
2026-02-14 14:47:19.507764 | TASK [ocp_agent_installer : Copy machine_configs to manifests dir]
2026-02-14 14:47:19.953448 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:19.966964 | 
2026-02-14 14:47:19.967268 | TASK [ocp_agent_installer : Copy config assets to manifests dir]
2026-02-14 14:47:20.443800 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:20.457088 | 
2026-02-14 14:47:20.457225 | TASK [ocp_agent_installer : Run tasks/pxe_assets.yml]
2026-02-14 14:47:20.506106 | hotstack | ok
2026-02-14 14:47:20.526535 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/pxe_assets.yml
2026-02-14 14:47:20.560421 | 
2026-02-14 14:47:20.560518 | TASK [ocp_agent_installer : Create the boot-artifacts directory]
2026-02-14 14:47:20.923516 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:47:20.941797 | 
2026-02-14 14:47:20.942237 | TASK [ocp_agent_installer : Create PXE assets]
2026-02-14 14:47:22.894458 | controller-0 | level=warning msg=Compute[0].Hyperthreading: Disabled is ignored
2026-02-14 14:47:23.806521 | controller-0 | level=warning msg=ControlPlane.Hyperthreading: Disabled is ignored
2026-02-14 14:47:23.806564 | controller-0 | level=info msg=Configuration has 1 master replicas and 0 worker replicas
2026-02-14 14:47:23.806578 | controller-0 | level=info msg=The rendezvous host IP (node0 IP) is 192.168.32.10
2026-02-14 14:47:24.444966 | controller-0 | level=info msg=Extracting base ISO from release payload
2026-02-14 14:47:57.295973 | controller-0 | level=info msg=Base ISO obtained from release and cached at [/home/zuul/.cache/agent/image_cache/coreos-x86_64.iso]
2026-02-14 14:48:53.004866 | controller-0 | level=info msg=Consuming Agent Config from target directory
2026-02-14 14:48:53.005343 | controller-0 | level=info msg=Consuming Extra Manifests from target directory
2026-02-14 14:48:53.504577 | controller-0 | level=info msg=Consuming Install Config from target directory
2026-02-14 14:48:53.504645 | controller-0 | level=info msg=Created iPXE script agent.x86_64.ipxe in boot-artifacts directory
2026-02-14 14:48:53.504723 | controller-0 | level=info msg=PXE boot artifacts created in: boot-artifacts
2026-02-14 14:48:53.504815 | controller-0 | level=info msg=Kernel parameters for PXE boot: rw ignition.firstboot ignition.platform.id=metal
2026-02-14 14:48:54.096531 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:48:54.109431 | 
2026-02-14 14:48:54.109570 | TASK [ocp_agent_installer : Set serial console in ipxe]
2026-02-14 14:48:54.787432 | hotstack -> 38.102.83.12 | changed: line replaced
2026-02-14 14:48:54.793751 | 
2026-02-14 14:48:54.793818 | TASK [ocp_agent_installer : Disable net.ifnames]
2026-02-14 14:48:55.306596 | hotstack -> 38.102.83.12 | changed: line replaced
2026-02-14 14:48:55.313250 | 
2026-02-14 14:48:55.313324 | LOOP [ocp_agent_installer : Copy boot-artifacts to the web server - (ocp_agent_installer_boot_artifacts_dir)]
2026-02-14 14:48:56.107783 | hotstack -> 38.102.83.12 | changed: "agent.x86_64-vmlinuz"
2026-02-14 14:48:56.108208 | 
2026-02-14 14:48:57.299416 | hotstack -> 38.102.83.12 | changed: "agent.x86_64-initrd.img"
2026-02-14 14:49:19.270574 | hotstack -> 38.102.83.12 | changed: "agent.x86_64-rootfs.img"
2026-02-14 14:49:19.691181 | hotstack -> 38.102.83.12 | changed: "agent.x86_64.ipxe"
2026-02-14 14:49:19.710361 | 
2026-02-14 14:49:19.710553 | TASK [ocp_agent_installer : Run tasks/iso_assets.yml]
2026-02-14 14:49:19.738125 | hotstack | skipping: Conditional result was False
2026-02-14 14:49:19.750669 | 
2026-02-14 14:49:19.750802 | TASK [ocp_agent_installer : Copy auth/kubeconfig to ~/.kube/config]
2026-02-14 14:49:20.206898 | hotstack -> 38.102.83.12 | changed
2026-02-14 14:49:20.220091 | 
2026-02-14 14:49:20.220234 | TASK [ocp_agent_installer : Wait for bootstrap-complete]
2026-02-14 14:49:57.829530 | controller-0 | level=info msg=Waiting for cluster install to initialize. Sleeping for 30 seconds
2026-02-14 14:50:29.833228 | controller-0 | level=info msg=Waiting for cluster install to initialize. Sleeping for 30 seconds
2026-02-14 14:51:01.838760 | controller-0 | level=info msg=Waiting for cluster install to initialize. Sleeping for 30 seconds
2026-02-14 14:51:36.903356 | controller-0 | level=info msg=Cannot access Rendezvous Host. There may be a network configuration problem, check console for additional info
2026-02-14 14:52:08.906906 | controller-0 | level=info msg=Cannot access Rendezvous Host. There may be a network configuration problem, check console for additional info
2026-02-14 14:52:40.930936 | controller-0 | level=info msg=Cluster is not ready for install. Check validations
2026-02-14 14:52:40.931200 | controller-0 | level=warning msg=Cluster validation: Single-node clusters must have a single control plane node and no workers.
2026-02-14 14:52:40.935280 | controller-0 | level=info msg=Host b45c7fed-78fe-402e-897b-9e9a5cd87be0: Successfully registered
2026-02-14 14:52:47.370185 | controller-0 | level=warning msg=Host master-0 validation: Host couldn't synchronize with any NTP server
2026-02-14 14:52:47.374176 | controller-0 | level=warning msg=Host master-0: updated status from discovering to insufficient (Host cannot be installed due to following failing validation(s): Host couldn't synchronize with any NTP server)
2026-02-14 14:52:51.403667 | controller-0 | level=warning msg=Cluster validation: The cluster has hosts that are not ready to install.
2026-02-14 14:52:51.403899 | controller-0 | level=info msg=Cluster validation: The cluster has the exact amount of dedicated control plane nodes.
2026-02-14 14:53:43.814975 | controller-0 | level=info msg=Host master-0 validation: Host NTP is synced
2026-02-14 14:53:43.817502 | controller-0 | level=info msg=Host master-0: updated status from insufficient to known (Host is ready to be installed)
2026-02-14 14:53:51.885250 | controller-0 | level=info msg=Cluster is ready for install
2026-02-14 14:53:51.885604 | controller-0 | level=info msg=Cluster validation: All hosts in the cluster are ready to install.
2026-02-14 14:53:55.918580 | controller-0 | level=info msg=Preparing cluster for installation
2026-02-14 14:53:58.512428 | controller-0 | level=info msg=Host master-0: updated status from known to preparing-for-installation (Host finished successfully to prepare for installation)
2026-02-14 14:54:57.069181 | controller-0 | level=info msg=Host master-0: New image status quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a9f59df0ef23d82f6521ed0d2df4f525b6a22d01f2bf0f570cf6e8032a7b3b14. result: success. time: 1.56 seconds; size: 448.78 Megabytes; download rate: 301.31 MBps
2026-02-14 14:55:03.647265 | controller-0 | level=info msg=Host master-0: updated status from preparing-for-installation to preparing-successful (Host finished successfully to prepare for installation)
2026-02-14 14:55:11.708126 | controller-0 | level=info msg=Cluster installation in progress
2026-02-14 14:55:19.791166 | controller-0 | level=info msg=Host master-0: updated status from preparing-successful to installing (Installation is in progress)
2026-02-14 14:55:43.984701 | controller-0 | level=info msg=Host: master-0, reached installation stage Installing: bootstrap
2026-02-14 14:55:50.038784 | controller-0 | level=info msg=Host: master-0, reached installation stage Waiting for bootkube
2026-02-14 14:57:07.775015 | controller-0 | level=info msg=Bootstrap Kube API Initialized
2026-02-14 14:59:35.730943 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk
2026-02-14 14:59:37.747034 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 7%
2026-02-14 14:59:39.765230 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 12%
2026-02-14 14:59:41.782280 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 18%
2026-02-14 14:59:43.797942 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 25%
2026-02-14 14:59:45.816429 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 32%
2026-02-14 14:59:47.832962 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 37%
2026-02-14 14:59:51.870575 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 45%
2026-02-14 14:59:53.888203 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 55%
2026-02-14 14:59:55.903854 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 60%
2026-02-14 14:59:57.923552 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 68%
2026-02-14 15:00:01.958724 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 75%
2026-02-14 15:00:03.974131 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 80%
2026-02-14 15:00:05.993938 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 87%
2026-02-14 15:00:08.007902 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 95%
2026-02-14 15:00:10.027568 | controller-0 | level=info msg=Host: master-0, reached installation stage Writing image to disk: 100%
2026-02-14 15:02:04.515871 | controller-0 | level=info msg=Bootstrap configMap status is complete
2026-02-14 15:02:06.516743 | controller-0 | level=info msg=Bootstrap is complete
2026-02-14 15:02:06.520685 | controller-0 | level=info msg=cluster bootstrap is complete
2026-02-14 15:02:06.877899 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:02:06.886239 | 
2026-02-14 15:02:06.886316 | TASK [ocp_agent_installer : Wait for install-complete]
2026-02-14 15:02:07.719245 | controller-0 | level=info msg=Bootstrap Kube API Initialized
2026-02-14 15:02:07.722188 | controller-0 | level=info msg=Bootstrap configMap status is complete
2026-02-14 15:02:09.722454 | controller-0 | level=info msg=Bootstrap is complete
2026-02-14 15:02:09.722555 | controller-0 | level=info msg=cluster bootstrap is complete
2026-02-14 15:25:35.641628 | controller-0 | level=info msg=Cluster is installed
2026-02-14 15:25:35.644301 | controller-0 | level=info msg=Install complete!
2026-02-14 15:25:35.644425 | controller-0 | level=info msg=To access the cluster as the system:admin user when using 'oc', run
2026-02-14 15:25:35.644461 | controller-0 | level=info msg=    export KUBECONFIG=/home/zuul/ocp-cluster/auth/kubeconfig
2026-02-14 15:25:35.644515 | controller-0 | level=info msg=Access the OpenShift web-console here: https://console-openshift-console.apps.sno.openstack.lab
2026-02-14 15:25:35.644565 | controller-0 | level=info msg=Login to the console with user: "kubeadmin", and password: "GGDBK-mEKcg-5boaX-VNhge"
2026-02-14 15:25:36.127630 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:25:36.139705 | 
2026-02-14 15:25:36.139891 | TASK [ocp_agent_installer : Add ingress certificate to CA trust]
2026-02-14 15:25:36.198966 | hotstack | ok
2026-02-14 15:25:36.216302 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/ocp_agent_installer/tasks/ingress_cert.yml
2026-02-14 15:25:36.258840 | 
2026-02-14 15:25:36.258988 | TASK [ocp_agent_installer : Extract ingress CA cert]
2026-02-14 15:25:37.784511 | hotstack -> 38.102.83.12 | Output suppressed because no_log was given
2026-02-14 15:25:37.794987 | 
2026-02-14 15:25:37.795159 | TASK [ocp_agent_installer : Write ingress cert to ca-trust]
2026-02-14 15:25:38.925861 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:25:38.940216 | 
2026-02-14 15:25:38.940421 | TASK [ocp_agent_installer : Update CA trust]
2026-02-14 15:25:41.475499 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:25:41.490404 | 
2026-02-14 15:25:41.490565 | TASK [ocp_agent_installer : Wait for 25 hours to prepare for snapshot]
2026-02-14 15:25:41.530445 | hotstack | skipping: Conditional result was False
2026-02-14 15:25:41.544299 | 
2026-02-14 15:25:41.544452 | TASK [ocp_agent_installer : Prepare for Snapshot - and shutdown]
2026-02-14 15:25:42.079150 | hotstack | skipping: Conditional result was False
2026-02-14 15:25:42.093476 | 
2026-02-14 15:25:42.093626 | TASK [ocp_agent_installer : Wait for cluster, uncordon, wait for cluster, wait for route API version ...]
2026-02-14 15:25:42.638395 | hotstack | skipping: Conditional result was False
2026-02-14 15:25:42.661867 | 
2026-02-14 15:25:42.662192 | TASK [Set up RedFish Virtual bmc]
2026-02-14 15:25:42.725115 | hotstack | Output suppressed because no_log was given
2026-02-14 15:25:42.839687 | 
2026-02-14 15:25:42.839812 | TASK [redfish_virtual_bmc : Create tempdir]
2026-02-14 15:25:43.325087 | hotstack | changed
2026-02-14 15:25:43.330685 | 
2026-02-14 15:25:43.330753 | TASK [redfish_virtual_bmc : Install package requirements for agent installer]
2026-02-14 15:25:46.045408 | hotstack | changed
2026-02-14 15:25:46.058180 | 
2026-02-14 15:25:46.058333 | TASK [redfish_virtual_bmc : Generate htpasswd]
2026-02-14 10:25:46.374461 | hotstack | admin:$2y$05$Yiy02j7vwLlrHy03CaegD.32UAWJzGeZgJMzCe.G8CZjeQB1/OOfe
2026-02-14 10:25:46.374607 | hotstack |
2026-02-14 15:25:46.611145 | hotstack | changed
2026-02-14 15:25:46.625137 | 
2026-02-14 15:25:46.625325 | LOOP [redfish_virtual_bmc : Copy files to tempdir]
2026-02-14 15:25:47.380388 | hotstack | changed: "namespace.yaml"
2026-02-14 15:25:48.063143 | hotstack | changed: "deployment.yaml"
2026-02-14 15:25:48.731579 | hotstack | changed: "service.yaml"
2026-02-14 15:25:48.746862 | 
2026-02-14 15:25:48.747084 | TASK [redfish_virtual_bmc : Template the ConfigMap]
2026-02-14 15:25:49.477077 | hotstack | changed
2026-02-14 15:25:49.490176 | 
2026-02-14 15:25:49.490315 | TASK [redfish_virtual_bmc : Template the Route]
2026-02-14 15:25:50.203555 | hotstack | changed
2026-02-14 15:25:50.216145 | 
2026-02-14 15:25:50.216302 | TASK [redfish_virtual_bmc : Load automation vars]
2026-02-14 15:25:50.261717 | hotstack | ok
2026-02-14 15:25:50.270706 | 
2026-02-14 15:25:50.270804 | TASK [Run the hotloop]
2026-02-14 15:25:50.303228 | hotstack | ok
2026-02-14 15:25:50.339622 | 
2026-02-14 15:25:50.339731 | TASK [hotloop : Assert config is defined]
2026-02-14 15:25:50.446319 | hotstack -> 38.102.83.12 | ok: All assertions passed
2026-02-14 15:25:50.459389 | 
2026-02-14 15:25:50.459518 | TASK [hotloop : Load stages]
2026-02-14 15:25:51.090285 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:25:51.106563 | 
2026-02-14 15:25:51.106728 | LOOP [hotloop : Ensure directory exists]
2026-02-14 15:25:51.586489 | hotstack -> 38.102.83.12 | changed: "/home/zuul/manifests/sushy_emulator_manifests"
2026-02-14 15:25:51.586962 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 15:25:51.587048 | 
2026-02-14 15:25:51.611990 | 
2026-02-14 15:25:51.612240 | TASK [hotloop : Create temporary hotloop work directory]
2026-02-14 15:25:52.079048 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:25:52.094230 | 
2026-02-14 15:25:52.094384 | TASK [hotloop : Create temporary directory for templates]
2026-02-14 15:25:52.388189 | hotstack -> localhost | changed
2026-02-14 15:25:52.403796 | 
2026-02-14 15:25:52.403979 | TASK [hotloop : Remove target directory for clean sync (local connection)]
2026-02-14 15:25:52.472411 | hotstack | skipping: Conditional result was False
2026-02-14 15:25:52.487413 | 
2026-02-14 15:25:52.487577 | TASK [hotloop : Copy work files (local connection)]
2026-02-14 15:25:52.572526 | hotstack | skipping: Conditional result was False
2026-02-14 15:25:52.580720 | 
2026-02-14 15:25:52.580834 | TASK [hotloop : Sync work files (remote connection) - using synchronize push]
2026-02-14 15:25:53.441990 | hotstack -> 38.102.83.115 | changed:
2026-02-14 15:25:53.442367 | hotstack -> 38.102.83.115 | .d..t...... ./
2026-02-14 15:25:53.442425 | hotstack -> 38.102.83.115 | <f+++++++++ config_map.yaml
2026-02-14 15:25:53.442468 | hotstack -> 38.102.83.115 | <f+++++++++ deployment.yaml
2026-02-14 15:25:53.442507 | hotstack -> 38.102.83.115 | <f+++++++++ namespace.yaml
2026-02-14 15:25:53.442577 | hotstack -> 38.102.83.115 | <f+++++++++ route.yaml
2026-02-14 15:25:53.442625 | hotstack -> 38.102.83.115 | <f+++++++++ service.yaml
2026-02-14 15:25:53.457167 | 
2026-02-14 15:25:53.457358 | LOOP [hotloop : Render all stage task files]
2026-02-14 15:25:53.990522 | hotstack -> localhost | ok:
2026-02-14 15:25:53.990951 | hotstack -> localhost | {
2026-02-14 15:25:53.991052 | hotstack -> localhost |   "manifest": "namespace.yaml",
2026-02-14 15:25:53.991106 | hotstack -> localhost |   "name": "Sushy Emulator : Namespace",
2026-02-14 15:25:53.991144 | hotstack -> localhost |   "wait_conditions": [
2026-02-14 15:25:53.991181 | hotstack -> localhost |     "oc wait namespaces sushy-emulator --for jsonpath='{.status.phase}'=Active --timeout=300s"
2026-02-14 15:25:53.991218 | hotstack -> localhost |   ]
2026-02-14 15:25:53.991256 | hotstack -> localhost | }
2026-02-14 15:25:54.485277 | hotstack -> localhost | ok:
2026-02-14 15:25:54.485424 | hotstack -> localhost | {
2026-02-14 15:25:54.485474 | hotstack -> localhost |   "name": "Sushy Emulator : Openstack clouds secret",
2026-02-14 15:25:54.485519 | hotstack -> localhost |   "shell": "oc create -n sushy-emulator secret generic os-client-config --save-config --dry-run=client --from-file=openstack-clouds-yaml=/home/zuul/.hotcloud/clouds.yaml --from-file=certificate-pem=/home/zuul/.hotcloud/cacert.pem --type=Opaque -o yaml | oc apply -f -"
2026-02-14 15:25:54.485558 | hotstack -> localhost | }
2026-02-14 15:25:55.085842 | hotstack -> localhost | ok:
2026-02-14 15:25:55.085938 | hotstack -> localhost | {
2026-02-14 15:25:55.085961 | hotstack -> localhost |   "manifest": "config_map.yaml",
2026-02-14 15:25:55.085981 | hotstack -> localhost |   "name": "Sushy Emulator : Config Map",
2026-02-14 15:25:55.085999 | hotstack -> localhost |   "wait_conditions": [
2026-02-14 15:25:55.086092 | hotstack -> localhost |     "oc wait -n sushy-emulator configmaps sushy-emulator-config --for jsonpath='{.metadata.name}'=sushy-emulator-config --timeout=30s"
2026-02-14 15:25:55.086121 | hotstack -> localhost |   ]
2026-02-14 15:25:55.086140 | hotstack -> localhost | }
2026-02-14 15:25:55.484531 | hotstack -> localhost | ok:
2026-02-14 15:25:55.484772 | hotstack -> localhost | {
2026-02-14 15:25:55.484831 | hotstack -> localhost |   "manifest": "deployment.yaml",
2026-02-14 15:25:55.484871 | hotstack -> localhost |   "name": "Sushy Emulator : Deployment",
2026-02-14 15:25:55.484920 | hotstack -> localhost |   "wait_conditions": [
2026-02-14 15:25:55.484956 | hotstack -> localhost |     "oc wait -n sushy-emulator pod -l app=sushy-emulator --for condition=Ready --timeout=300s"
2026-02-14 15:25:55.484991 | hotstack -> localhost |   ]
2026-02-14 15:25:55.485061 | hotstack -> localhost | }
2026-02-14 15:25:55.972765 | hotstack -> localhost | ok:
2026-02-14 15:25:55.972881 | hotstack -> localhost | {
2026-02-14 15:25:55.972904 | hotstack -> localhost |   "manifest": "service.yaml",
2026-02-14 15:25:55.972922 | hotstack -> localhost |   "name": "Sushy Emulator : Service",
2026-02-14 15:25:55.972939 | hotstack -> localhost |   "wait_conditions": [
2026-02-14 15:25:55.972954 | hotstack -> localhost |     "oc wait -n sushy-emulator service/sushy-emulator-service --for jsonpath='{.status.loadBalancer}' --timeout=30s"
2026-02-14 15:25:55.972970 | hotstack -> localhost |   ]
2026-02-14 15:25:55.972986 | hotstack -> localhost | }
2026-02-14 15:25:56.482511 | hotstack -> localhost | ok:
2026-02-14 15:25:56.482634 | hotstack -> localhost | {
2026-02-14 15:25:56.482659 | hotstack -> localhost |   "manifest": "route.yaml",
2026-02-14 15:25:56.482679 | hotstack -> localhost |   "name": "Sushy Emulator : Route",
2026-02-14 15:25:56.482697 | hotstack -> localhost |   "wait_conditions": [
2026-02-14 15:25:56.482714 | hotstack -> localhost |     "oc wait -n sushy-emulator routes/sushy-emulator-route --for jsonpath='{.status.ingress[*].conditions[*].status}'=True --timeout=30s"
2026-02-14 15:25:56.482731 | hotstack -> localhost |   ]
2026-02-14 15:25:56.482747 | hotstack -> localhost | }
2026-02-14 15:25:56.504678 | 
2026-02-14 15:25:56.504878 | LOOP [hotloop : Execute automation stages]
2026-02-14 15:25:56.545617 | hotstack | ok:
2026-02-14 15:25:56.545961 | hotstack | {
2026-02-14 15:25:56.545990 | hotstack |   "manifest": "namespace.yaml",
2026-02-14 15:25:56.546009 | hotstack |   "name": "Sushy Emulator : Namespace",
2026-02-14 15:25:56.546050 | hotstack |   "wait_conditions": [
2026-02-14 15:25:56.546067 | hotstack |     "oc wait namespaces sushy-emulator --for jsonpath='{.status.phase}'=Active --timeout=300s"
2026-02-14 15:25:56.546083 | hotstack |   ]
2026-02-14 15:25:56.546099 | hotstack | }
2026-02-14 15:25:56.546127 | hotstack | ok: All items complete
2026-02-14 15:25:56.546172 | 
2026-02-14 15:25:56.551994 | hotstack | ok:
2026-02-14 15:25:56.552080 | hotstack | {
2026-02-14 15:25:56.552104 | hotstack |   "name": "Sushy Emulator : Openstack clouds secret",
2026-02-14 15:25:56.552123 | hotstack |   "shell": "oc create -n sushy-emulator secret generic os-client-config --save-config --dry-run=client --from-file=openstack-clouds-yaml=/home/zuul/.hotcloud/clouds.yaml --from-file=certificate-pem=/home/zuul/.hotcloud/cacert.pem --type=Opaque -o yaml | oc apply -f -"
2026-02-14 15:25:56.552145 | hotstack | }
2026-02-14 15:25:56.558484 | hotstack | ok:
2026-02-14 15:25:56.558608 | hotstack | {
2026-02-14 15:25:56.558636 | hotstack |   "manifest": "config_map.yaml",
2026-02-14 15:25:56.558654 | hotstack |   "name": "Sushy Emulator : Config Map",
2026-02-14 15:25:56.558671 | hotstack |   "wait_conditions": [
2026-02-14 15:25:56.558688 | hotstack |     "oc wait -n sushy-emulator configmaps sushy-emulator-config --for jsonpath='{.metadata.name}'=sushy-emulator-config --timeout=30s"
2026-02-14 15:25:56.558704 | hotstack |   ]
2026-02-14 15:25:56.558720 | hotstack | }
2026-02-14 15:25:56.563710 | hotstack | ok:
2026-02-14 15:25:56.563754 | hotstack | {
2026-02-14 15:25:56.563776 | hotstack |   "manifest": "deployment.yaml",
2026-02-14 15:25:56.563793 | hotstack |   "name": "Sushy Emulator : Deployment",
2026-02-14 15:25:56.563810 | hotstack |   "wait_conditions": [
2026-02-14 15:25:56.563826 | hotstack |     "oc wait -n sushy-emulator pod -l app=sushy-emulator --for condition=Ready --timeout=300s"
2026-02-14 15:25:56.563842 | hotstack |   ]
2026-02-14 15:25:56.563857 | hotstack | }
2026-02-14 15:25:56.569242 | hotstack | ok:
2026-02-14 15:25:56.569307 | hotstack | {
2026-02-14 15:25:56.569339 | hotstack |   "manifest": "service.yaml",
2026-02-14 15:25:56.569379 | hotstack |   "name": "Sushy Emulator : Service",
2026-02-14 15:25:56.569399 | hotstack |   "wait_conditions": [
2026-02-14 15:25:56.569442 | hotstack |     "oc wait -n sushy-emulator service/sushy-emulator-service --for jsonpath='{.status.loadBalancer}' --timeout=30s"
2026-02-14 15:25:56.569460 | hotstack |   ]
2026-02-14 15:25:56.569475 | hotstack | }
2026-02-14 15:25:56.574777 | hotstack | ok:
2026-02-14 15:25:56.574829 | hotstack | {
2026-02-14 15:25:56.574851 | hotstack |   "manifest": "route.yaml",
2026-02-14 15:25:56.574874 | hotstack |   "name": "Sushy Emulator : Route",
2026-02-14 15:25:56.574891 | hotstack |   "wait_conditions": [
2026-02-14 15:25:56.574911 | hotstack |     "oc wait -n sushy-emulator routes/sushy-emulator-route --for jsonpath='{.status.ingress[*].conditions[*].status}'=True --timeout=30s"
2026-02-14 15:25:56.574928 | hotstack |   ]
2026-02-14 15:25:56.574948 | hotstack | }
2026-02-14 15:25:56.596502 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.byzd3eubhotloop_templates/stage_1.yml
2026-02-14 15:25:56.599046 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.byzd3eubhotloop_templates/stage_2.yml
2026-02-14 15:25:56.602584 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.byzd3eubhotloop_templates/stage_3.yml
2026-02-14 15:25:56.606597 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.byzd3eubhotloop_templates/stage_4.yml
2026-02-14 15:25:56.610011 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.byzd3eubhotloop_templates/stage_5.yml
2026-02-14 15:25:56.613758 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.byzd3eubhotloop_templates/stage_6.yml
2026-02-14 15:25:56.625388 | 
2026-02-14 15:25:56.625474 | TASK [hotloop : Stage: Sushy Emulator : Namespace :: Apply static manifest]
2026-02-14 15:25:56.658095 | hotstack | ok
2026-02-14 15:25:56.672591 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/hotloop/tasks/static_manifest.yml
2026-02-14 15:25:56.682543 | 
2026-02-14 15:25:56.682640 | TASK [hotloop : Stage: Sushy Emulator : Namespace :: Ensure manifest dir]
2026-02-14 15:25:57.012217 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:25:57.019734 | 
2026-02-14 15:25:57.019840 | TASK [hotloop : Stage: Sushy Emulator : Namespace :: Check if manifest exists in synced work directory]
2026-02-14 15:25:57.364603 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:25:57.380136 | 
2026-02-14 15:25:57.380321 | TASK [hotloop : Stage: Sushy Emulator : Namespace :: Copy manifest]
2026-02-14 15:25:57.913171 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:25:57.921812 | 
2026-02-14 15:25:57.921907 | LOOP [hotloop : Stage: Sushy Emulator : Namespace :: Apply patches]
2026-02-14 15:25:57.976260 | 
2026-02-14 15:25:57.976553 | TASK [hotloop : Stage: Sushy Emulator : Namespace :: Apply static manifest]
2026-02-14 15:25:58.575193 | hotstack -> 38.102.83.12 | changed: Manifest file /home/zuul/manifests/sushy_emulator_manifests/namespace.yaml applied and saved as /home/zuul/manifests/sushy_emulator_manifests/namespace.yaml.applied with log at /home/zuul/manifests/sushy_emulator_manifests/namespace.yaml.applied.log
2026-02-14 15:25:58.588277 | 
2026-02-14 15:25:58.588428 | LOOP [hotloop : Stage: Sushy Emulator : Namespace :: Wait conditions]
2026-02-14 15:25:59.227578 | hotstack -> 38.102.83.12 | ok: "oc wait namespaces sushy-emulator --for jsonpath='{.status.phase}'=Active --timeout=300s"
2026-02-14 15:25:59.228003 | 
2026-02-14 15:25:59.244190 | 
2026-02-14 15:25:59.244388 | TASK [hotloop : Stage: Sushy Emulator : Openstack clouds secret :: Run shell]
2026-02-14 15:25:59.781709 | controller-0 | secret/os-client-config created
2026-02-14 15:26:00.280160 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:26:00.293693 | 
2026-02-14 15:26:00.293961 | TASK [hotloop : Stage: Sushy Emulator : Config Map :: Apply static manifest]
2026-02-14 15:26:00.332212 | hotstack | ok
2026-02-14 15:26:00.352136 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/hotloop/tasks/static_manifest.yml
2026-02-14 15:26:00.364779 | 
2026-02-14 15:26:00.364873 | TASK [hotloop : Stage: Sushy Emulator : Config Map :: Ensure manifest dir]
2026-02-14 15:26:00.868990 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:00.886402 | 
2026-02-14 15:26:00.886585 | TASK [hotloop : Stage: Sushy Emulator : Config Map :: Check if manifest exists in synced work directory]
2026-02-14 15:26:01.384453 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:01.401492 | 
2026-02-14 15:26:01.401687 | TASK [hotloop : Stage: Sushy Emulator : Config Map :: Copy manifest]
2026-02-14 15:26:02.063826 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:26:02.081061 | 
2026-02-14 15:26:02.081259 | LOOP [hotloop : Stage: Sushy Emulator : Config Map :: Apply patches]
2026-02-14 15:26:02.134819 | 
2026-02-14 15:26:02.135149 | TASK [hotloop : Stage: Sushy Emulator : Config Map :: Apply static manifest]
2026-02-14 15:26:02.870388 | hotstack -> 38.102.83.12 | changed: Manifest file /home/zuul/manifests/sushy_emulator_manifests/config_map.yaml applied and saved as /home/zuul/manifests/sushy_emulator_manifests/config_map.yaml.applied with log at /home/zuul/manifests/sushy_emulator_manifests/config_map.yaml.applied.log
2026-02-14 15:26:02.883929 | 
2026-02-14 15:26:02.884191 | LOOP [hotloop : Stage: Sushy Emulator : Config Map :: Wait conditions]
2026-02-14 15:26:03.662570 | hotstack -> 38.102.83.12 | ok: "oc wait -n sushy-emulator configmaps sushy-emulator-config --for jsonpath='{.metadata.name}'=sushy-emulator-config --timeout=30s"
2026-02-14 15:26:03.663064 | 
2026-02-14 15:26:03.687355 | 
2026-02-14 15:26:03.687547 | TASK [hotloop : Stage: Sushy Emulator : Deployment :: Apply static manifest]
2026-02-14 15:26:03.728647 | hotstack | ok
2026-02-14 15:26:03.748662 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/hotloop/tasks/static_manifest.yml
2026-02-14 15:26:03.762067 | 
2026-02-14 15:26:03.762159 | TASK [hotloop : Stage: Sushy Emulator : Deployment :: Ensure manifest dir]
2026-02-14 15:26:04.276009 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:04.285285 | 
2026-02-14 15:26:04.285389 | TASK [hotloop : Stage: Sushy Emulator : Deployment :: Check if manifest exists in synced work directory]
2026-02-14 15:26:04.823816 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:04.842297 | 
2026-02-14 15:26:04.842474 | TASK [hotloop : Stage: Sushy Emulator : Deployment :: Copy manifest]
2026-02-14 15:26:05.555651 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:26:05.571346 | 
2026-02-14 15:26:05.571520 | LOOP [hotloop : Stage: Sushy Emulator : Deployment :: Apply patches]
2026-02-14 15:26:05.639346 | 
2026-02-14 15:26:05.639644 | TASK [hotloop : Stage: Sushy Emulator : Deployment :: Apply static manifest]
2026-02-14 15:26:06.414457 | hotstack -> 38.102.83.12 | changed: Manifest file /home/zuul/manifests/sushy_emulator_manifests/deployment.yaml applied and saved as /home/zuul/manifests/sushy_emulator_manifests/deployment.yaml.applied with log at /home/zuul/manifests/sushy_emulator_manifests/deployment.yaml.applied.log
2026-02-14 15:26:06.428928 | 
2026-02-14 15:26:06.429160 | LOOP [hotloop : Stage: Sushy Emulator : Deployment :: Wait conditions]
2026-02-14 15:26:51.098082 | hotstack -> 38.102.83.12 | ok: "oc wait -n sushy-emulator pod -l app=sushy-emulator --for condition=Ready --timeout=300s"
2026-02-14 15:26:51.098746 | 
2026-02-14 15:26:51.124784 | 
2026-02-14 15:26:51.124963 | TASK [hotloop : Stage: Sushy Emulator : Service :: Apply static manifest]
2026-02-14 15:26:51.161298 | hotstack | ok
2026-02-14 15:26:51.178738 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/hotloop/tasks/static_manifest.yml
2026-02-14 15:26:51.189300 | 
2026-02-14 15:26:51.189412 | TASK [hotloop : Stage: Sushy Emulator : Service :: Ensure manifest dir]
2026-02-14 15:26:51.543632 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:51.560550 | 
2026-02-14 15:26:51.560736 | TASK [hotloop : Stage: Sushy Emulator : Service :: Check if manifest exists in synced work directory]
2026-02-14 15:26:51.873775 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:51.890126 | 
2026-02-14 15:26:51.890303 | TASK [hotloop : Stage: Sushy Emulator : Service :: Copy manifest]
2026-02-14 15:26:52.305275 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:26:52.319990 | 
2026-02-14 15:26:52.320192 | LOOP [hotloop : Stage: Sushy Emulator : Service :: Apply patches]
2026-02-14 15:26:52.382205 | 
2026-02-14 15:26:52.382527 | TASK [hotloop : Stage: Sushy Emulator : Service :: Apply static manifest]
2026-02-14 15:26:52.879157 | hotstack -> 38.102.83.12 | changed: Manifest file /home/zuul/manifests/sushy_emulator_manifests/service.yaml applied and saved as /home/zuul/manifests/sushy_emulator_manifests/service.yaml.applied with log at /home/zuul/manifests/sushy_emulator_manifests/service.yaml.applied.log
2026-02-14 15:26:52.894463 | 
2026-02-14 15:26:52.894666 | LOOP [hotloop : Stage: Sushy Emulator : Service :: Wait conditions]
2026-02-14 15:26:53.414785 | hotstack -> 38.102.83.12 | ok: "oc wait -n sushy-emulator service/sushy-emulator-service --for jsonpath='{.status.loadBalancer}' --timeout=30s"
2026-02-14 15:26:53.415535 | 
2026-02-14 15:26:53.439147 | 
2026-02-14 15:26:53.439349 | TASK [hotloop : Stage: Sushy Emulator : Route :: Apply static manifest]
2026-02-14 15:26:53.480108 | hotstack | ok
2026-02-14 15:26:53.499276 | hotstack | included: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/trusted/project_3/github.com/openstack-k8s-operators/hotstack/roles/hotloop/tasks/static_manifest.yml
2026-02-14 15:26:53.512380 | 
2026-02-14 15:26:53.512514 | TASK [hotloop : Stage: Sushy Emulator : Route :: Ensure manifest dir]
2026-02-14 15:26:53.856861 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:53.873821 | 
2026-02-14 15:26:53.873981 | TASK [hotloop : Stage: Sushy Emulator : Route :: Check if manifest exists in synced work directory]
2026-02-14 15:26:54.167354 | hotstack -> 38.102.83.12 | ok
2026-02-14 15:26:54.183990 | 
2026-02-14 15:26:54.184202 | TASK [hotloop : Stage: Sushy Emulator : Route :: Copy manifest]
2026-02-14 15:26:54.580380 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:26:54.595329 | 
2026-02-14 15:26:54.595481 | LOOP [hotloop : Stage: Sushy Emulator : Route :: Apply patches]
2026-02-14 15:26:54.659167 | 
2026-02-14 15:26:54.659480 | TASK [hotloop : Stage: Sushy Emulator : Route :: Apply static manifest]
2026-02-14 15:26:55.106286 | hotstack -> 38.102.83.12 | ERROR
2026-02-14 15:26:55.106835 | hotstack -> 38.102.83.12 | {
2026-02-14 15:26:55.106907 | hotstack -> 38.102.83.12 |   "error": "",
2026-02-14 15:26:55.106950 | hotstack -> 38.102.83.12 |   "msg": "Error while applying manifest file /home/zuul/manifests/sushy_emulator_manifests/route.yaml. Saved to /home/zuul/manifests/sushy_emulator_manifests/route.yaml.20260214_152655.failed with logs in /home/zuul/manifests/sushy_emulator_manifests/route.yaml.20260214_152655.log",
2026-02-14 15:26:55.106989 | hotstack -> 38.102.83.12 |   "rc": 1,
2026-02-14 15:26:55.107075 | hotstack -> 38.102.83.12 |   "success": false
2026-02-14 15:26:55.107116 | hotstack -> 38.102.83.12 | }
failure
2026-02-14 15:26:55.120858 | 
2026-02-14 15:26:55.120991 | TASK [redfish_virtual_bmc : Clean up the tempdir]
2026-02-14 15:26:55.540321 | hotstack | changed
2026-02-14 15:26:55.543725 | 
2026-02-14 15:26:55.543828 | PLAY RECAP
2026-02-14 15:26:55.543912 | hotstack | ok: 132 changed: 77 unreachable: 0 failed: 1 skipped: 21 rescued: 0 ignored: 0
2026-02-14 15:26:55.543955 | 
2026-02-14 15:26:55.721159 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/hotstack/pre_stage2.yaml@master]
2026-02-14 15:26:55.724646 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/hotstack/post.yaml@master]
2026-02-14 15:26:56.318315 | 
2026-02-14 15:26:56.318426 | PLAY [Collect hotstack logs and clean up environment]
2026-02-14 15:26:56.342555 | 
2026-02-14 15:26:56.342648 | TASK [Fetch stack outputs from remote to zuul executor]
2026-02-14 15:26:56.940670 | hotstack | Output suppressed because no_log was given
2026-02-14 15:26:56.953277 | 
2026-02-14 15:26:56.953458 | TASK [Load stack output vars with template resolution]
2026-02-14 15:26:56.996713 | hotstack | Output suppressed because no_log was given
2026-02-14 15:26:57.002384 | 
2026-02-14 15:26:57.002459 | TASK [Add Hotstack controller-0 to ansible inventory]
2026-02-14 15:26:57.050825 | hotstack | Output suppressed because no_log was given
2026-02-14 15:26:57.057437 | 
2026-02-14 15:26:57.057516 | TASK [Wait for Hotstack controller-0 to be ready]
2026-02-14 15:27:02.661560 | hotstack -> 38.102.83.12 | Output suppressed because no_log was given
2026-02-14 15:27:02.675182 | 
2026-02-14 15:27:02.675372 | TASK [Collect hotstack logs]
2026-02-14 15:27:02.704055 | hotstack | Output suppressed because no_log was given
2026-02-14 15:27:02.757790 | 
2026-02-14 15:27:02.757921 | TASK [hotlogs : Initialize failure tracking]
2026-02-14 15:27:02.779834 | hotstack | ok
2026-02-14 15:27:02.787255 | 
2026-02-14 15:27:02.787346 | TASK [hotlogs : Ensure logs directory]
2026-02-14 15:27:03.227865 | hotstack | changed
2026-02-14 15:27:03.245706 | 
2026-02-14 15:27:03.245860 | LOOP [hotlogs : Create hotlogs sub dirs]
2026-02-14 15:27:03.485503 | hotstack | changed:
2026-02-14 15:27:03.485744 | hotstack | {
2026-02-14 15:27:03.485777 | hotstack |   "dest_dir": "ocp_cluster",
2026-02-14 15:27:03.485799 | hotstack |   "src": "/home/zuul/ocp-cluster/.openshift_install.log"
2026-02-14 15:27:03.485820 | hotstack | }
2026-02-14 15:27:03.696887 | hotstack | ok:
2026-02-14 15:27:03.696983 | hotstack | {
2026-02-14 15:27:03.697007 | hotstack |   "dest_dir": "ocp_cluster",
2026-02-14 15:27:03.697053 | hotstack |   "src": "/home/zuul/cluster-custom-config/"
2026-02-14 15:27:03.697075 | hotstack | }
2026-02-14 15:27:03.914886 | hotstack | changed:
2026-02-14 15:27:03.914999 | hotstack | {
2026-02-14 15:27:03.915063 | hotstack |   "dest_dir": "data",
2026-02-14 15:27:03.915090 | hotstack |   "src": "/home/zuul/data/"
2026-02-14 15:27:03.915114 | hotstack | }
2026-02-14 15:27:04.142963 | hotstack | changed:
2026-02-14 15:27:04.143119 | hotstack | {
2026-02-14 15:27:04.143146 | hotstack |   "dest_dir": "manifests",
2026-02-14 15:27:04.143165 | hotstack |   "src": "/home/zuul/manifests/"
2026-02-14 15:27:04.143182 | hotstack | }
2026-02-14 15:27:04.372521 | hotstack | ok:
2026-02-14 15:27:04.372757 | hotstack | {
2026-02-14 15:27:04.372817 | hotstack |   "dest_dir": ".",
2026-02-14 15:27:04.372862 | hotstack |   "src": "/home/zuul/must-gather.tar.gz"
2026-02-14 15:27:04.372902 | hotstack | }
2026-02-14 15:27:04.554534 | hotstack | ok:
2026-02-14 15:27:04.554685 | hotstack | {
2026-02-14 15:27:04.554739 | hotstack |   "dest_dir": ".",
2026-02-14 15:27:04.554781 | hotstack |   "src": "/home/zuul/logs/tempest/"
2026-02-14 15:27:04.554819 | hotstack | }
2026-02-14 15:27:04.573485 | 
2026-02-14 15:27:04.573711 | TASK [hotlogs : Run must-gather operations]
2026-02-14 15:27:06.967232 | hotstack -> 38.102.83.12 | ERROR
2026-02-14 15:27:06.967598 | hotstack -> 38.102.83.12 | {
2026-02-14 15:27:06.967662 | hotstack -> 38.102.83.12 |   "msg": "Must-gather failed: Error from server (Forbidden): pods \"must-gather-\" is forbidden: error looking up service account openshift-must-gather-m8qzc/default: serviceaccount \"default\" not found\nError from server (Forbidden): pods \"must-gather-\" is forbidden: error looking up service account openshift-must-gather-m8qzc/default: serviceaccount \"default\" not found\n (Command: oc adm must-gather --image-stream=openshift/must-gather --image=quay.io/openstack-k8s-operators/openstack-must-gather --dest-dir=/home/zuul/must-gather --timeout=10m --host-network=false -- ADDITIONAL_NAMESPACES=sushy-emulator OPENSTACK_DATABASES=ALL SOS_EDPM=all SOS_DECOMPRESS=0 gather)",
2026-02-14 15:27:06.967718 | hotstack -> 38.102.83.12 |   "rc": 1
2026-02-14 15:27:06.967758 | hotstack -> 38.102.83.12 | }
2026-02-14 15:27:06.982406 | 
2026-02-14 15:27:06.982691 | TASK [hotlogs : Display must-gather failure]
2026-02-14 15:27:07.062162 | hotstack | ok:
2026-02-14 15:27:07.062444 | hotstack | Must-gather operations failed: Must-gather failed: Error from server (Forbidden): pods "must-gather-" is forbidden: error looking up service account openshift-must-gather-m8qzc/default: serviceaccount "default" not found
2026-02-14 15:27:07.062533 | hotstack | Error from server (Forbidden): pods "must-gather-" is forbidden: error looking up service account openshift-must-gather-m8qzc/default: serviceaccount "default" not found
2026-02-14 15:27:07.062583 | hotstack |  (Command: oc adm must-gather --image-stream=openshift/must-gather --image=quay.io/openstack-k8s-operators/openstack-must-gather --dest-dir=/home/zuul/must-gather --timeout=10m --host-network=false -- ADDITIONAL_NAMESPACES=sushy-emulator OPENSTACK_DATABASES=ALL SOS_EDPM=all SOS_DECOMPRESS=0 gather)
2026-02-14 15:27:07.073854 | 
2026-02-14 15:27:07.073985 | TASK [hotlogs : Track must-gather failure]
2026-02-14 15:27:07.149643 | hotstack | ok
2026-02-14 15:27:07.159453 | 
2026-02-14 15:27:07.159579 | LOOP [hotlogs : Collect tempest logs from test-operator PVCs]
2026-02-14 15:27:07.740905 | hotstack -> 38.102.83.12 | changed
2026-02-14 15:27:07.741356 | hotstack -> 38.102.83.12 | changed: All items complete
2026-02-14 15:27:07.741415 | 
2026-02-14 15:27:07.753788 | 
2026-02-14 15:27:07.753936 | LOOP [hotlogs : Display tempest log collection results]
2026-02-14 15:27:07.823365 | hotstack | ok: Namespace openstack: ['==========================================', 'Tempest Log Collection', '==========================================', 'Namespace: openstack', 'Logs directory: /home/zuul/logs/tempest', '==========================================', '', 'Searching for tempest log PVCs...', 'WARNING: No tempest PVCs found', "This is normal if tests haven't run yet or logs were already collected"]
2026-02-14 15:27:07.843577 | 
2026-02-14 15:27:07.843763 | LOOP [hotlogs : Track tempest log collection failures]
2026-02-14 15:27:07.875298 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:07.875788 | hotstack | ok: All items complete
2026-02-14 15:27:07.875846 | 
2026-02-14 15:27:07.895534 | 
2026-02-14 15:27:07.895716 | LOOP [hotlogs : Collect hotstack data]
2026-02-14 15:27:08.527107 | hotstack -> 38.102.83.115 | changed: >f+++++++++ .openshift_install.log
2026-02-14 15:27:08.527351 | 
2026-02-14 15:27:09.018686 | hotstack -> 38.102.83.115 | changed:
2026-02-14 15:27:09.018803 | hotstack -> 38.102.83.115 | .d..t...... ./
2026-02-14 15:27:09.018828 | hotstack -> 38.102.83.115 | cd+++++++++ butane/
2026-02-14 15:27:09.018847 | hotstack -> 38.102.83.115 | >f+++++++++ butane/90-master-disable-netifnames.bu
2026-02-14 15:27:09.018866 | hotstack -> 38.102.83.115 | >f+++++++++ butane/90-master-enable-iscsi.bu
2026-02-14 15:27:09.018884 | hotstack -> 38.102.83.115 | >f+++++++++ butane/91-master-enable-multipath.bu
2026-02-14 15:27:09.018902 | hotstack -> 38.102.83.115 | >f+++++++++ butane/92-master-lv-cinder-volumes.bu
2026-02-14 15:27:09.018919 | hotstack -> 38.102.83.115 | cd+++++++++ config-assets/
2026-02-14 15:27:09.018935 | hotstack -> 38.102.83.115 | >f+++++++++ config-assets/95-etcd_config.yaml
2026-02-14 15:27:09.018952 | hotstack -> 38.102.83.115 | >f+++++++++ config-assets/ovn_k8s_config.yaml
2026-02-14 15:27:09.018968 | hotstack -> 38.102.83.115 | cd+++++++++ machine-configs/
2026-02-14 15:27:09.018985 | hotstack -> 38.102.83.115 | >f+++++++++ machine-configs/90-master-disable-netifnames.yaml
2026-02-14 15:27:09.019002 | hotstack -> 38.102.83.115 | >f+++++++++ machine-configs/90-master-enable-iscsi.yaml
2026-02-14 15:27:09.019039 | hotstack -> 38.102.83.115 | >f+++++++++ machine-configs/91-master-enable-multipath.yaml
2026-02-14 15:27:09.019058 | hotstack -> 38.102.83.115 | >f+++++++++ machine-configs/92-master-lv-cinder-volumes.yaml
2026-02-14 15:27:09.496803 | hotstack -> 38.102.83.115 | changed:
2026-02-14 15:27:09.496916 | hotstack -> 38.102.83.115 | .d..t...... ./
2026-02-14 15:27:09.496940 | hotstack -> 38.102.83.115 | >f+++++++++ ironic_nodes.yaml
2026-02-14 15:27:10.022659 | hotstack -> 38.102.83.115 | changed:
2026-02-14 15:27:10.022781 | hotstack -> 38.102.83.115 | .d..t...... ./
2026-02-14 15:27:10.022804 | hotstack -> 38.102.83.115 | cd+++++++++ sushy_emulator_manifests/
2026-02-14 15:27:10.022822 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/config_map.yaml.applied
2026-02-14 15:27:10.022840 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/config_map.yaml.applied.log
2026-02-14 15:27:10.022857 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/deployment.yaml.applied
2026-02-14 15:27:10.022873 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/deployment.yaml.applied.log
2026-02-14 15:27:10.022889 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/namespace.yaml.applied
2026-02-14 15:27:10.022905 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/namespace.yaml.applied.log
2026-02-14 15:27:10.022924 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/route.yaml.20260214_152655.failed
2026-02-14 15:27:10.022946 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/route.yaml.20260214_152655.log
2026-02-14 15:27:10.022969 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/service.yaml.applied
2026-02-14 15:27:10.022988 | hotstack -> 38.102.83.115 | >f+++++++++ sushy_emulator_manifests/service.yaml.applied.log
2026-02-14 15:27:10.640206 | hotstack -> 38.102.83.115 | ERROR: Item: {'src': '/home/zuul/must-gather.tar.gz', 'dest_dir': '.'}
2026-02-14 15:27:10.640383 | hotstack -> 38.102.83.115 | {
2026-02-14 15:27:10.640412 | hotstack -> 38.102.83.115 |   "ansible_loop_var": "item",
2026-02-14 15:27:10.640432 | hotstack -> 38.102.83.115 |   "item": {
2026-02-14 15:27:10.640449 | hotstack -> 38.102.83.115 |     "dest_dir": ".",
2026-02-14 15:27:10.640466 | hotstack -> 38.102.83.115 |     "src": "/home/zuul/must-gather.tar.gz"
2026-02-14 15:27:10.640487 | hotstack -> 38.102.83.115 |   },
2026-02-14 15:27:10.640508 | hotstack -> 38.102.83.115 |   "msg": "Warning: Permanently added '38.102.83.12' (ED25519) to the list of known hosts.\r\nrsync: [sender] link_stat \"/home/zuul/must-gather.tar.gz\" failed: No such file or directory (2)\nrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1841) [Receiver=3.2.5]\nrsync: [Receiver] write error: Broken pipe (32)\n",
2026-02-14 15:27:10.640529 | hotstack -> 38.102.83.115 |   "rc": 23
2026-02-14 15:27:10.640548 | hotstack -> 38.102.83.115 | }
2026-02-14 15:27:11.150888 | hotstack -> 38.102.83.115 | changed: .d..t...... ./
2026-02-14 15:27:11.162375 | hotstack | ERROR: Ignoring Errors
2026-02-14 15:27:11.175498 | 
2026-02-14 15:27:11.175620 | LOOP [hotlogs : Check hotstack data collection failures]
2026-02-14 15:27:11.246249 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:11.252128 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:11.258811 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:11.265566 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:11.283669 | hotstack | ERROR
2026-02-14 15:27:11.283867 | hotstack | {
2026-02-14 15:27:11.283924 | hotstack |   "msg": "Unexpected templating type error occurred on ({{ hotlogs_failures + ['Failed to collect: ' + item.item] }}): can only concatenate str (not \"dict\") to str. can only concatenate str (not \"dict\") to str"
2026-02-14 15:27:11.283964 | hotstack | }
2026-02-14 15:27:11.293712 | 
2026-02-14 15:27:11.293817 | TASK [Print message that log collection failed]
2026-02-14 15:27:11.327221 | hotstack | ok: WARNING: Log collection failed and may be incomplete. Continue with cleanup ...
2026-02-14 15:27:11.334036 | 
2026-02-14 15:27:11.334127 | TASK [Set _auth_type fact]
2026-02-14 15:27:11.376167 | hotstack | Output suppressed because no_log was given
2026-02-14 15:27:11.384906 | 
2026-02-14 15:27:11.385000 | TASK [Write clouds.yaml]
2026-02-14 15:27:12.388232 | hotstack | Output suppressed because no_log was given
2026-02-14 15:27:12.395461 | 
2026-02-14 15:27:12.395589 | TASK [Delete application credentials]
2026-02-14 15:27:13.967997 | hotstack | changed
2026-02-14 15:27:13.978922 | 
2026-02-14 15:27:13.979083 | TASK [Print message that application credential failed to delete]
2026-02-14 15:27:14.006053 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:14.017808 | 
2026-02-14 15:27:14.017937 | TASK [Print message that application credential deleted successfully]
2026-02-14 15:27:14.054904 | hotstack | ok: Application credential deleted successfully.
2026-02-14 15:27:14.066009 | 
2026-02-14 15:27:14.066170 | TASK [Write clouds.yaml with application credentials]
2026-02-14 15:27:14.105268 | 
2026-02-14 15:27:14.105544 | TASK [Delete the heat stack]
2026-02-14 10:27:46.645607 | hotstack | 2026-02-14 15:27:16Z [hs-e16e435992bd479db8414c1353b1e0de]: DELETE_IN_PROGRESS  Stack DELETE started
2026-02-14 10:27:46.645744 | hotstack | 2026-02-14 15:27:16Z [hs-e16e435992bd479db8414c1353b1e0de.controller]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645755 | hotstack | 2026-02-14 15:27:17Z [hs-e16e435992bd479db8414c1353b1e0de.ctlplane-net-router-interface]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645760 | hotstack | 2026-02-14 15:27:18Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645765 | hotstack | 2026-02-14 15:27:19Z [hs-e16e435992bd479db8414c1353b1e0de.ctlplane-net-router-interface]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645770 | hotstack | 2026-02-14 15:27:19Z [hs-e16e435992bd479db8414c1353b1e0de.master0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645774 | hotstack | 2026-02-14 15:27:20Z [hs-e16e435992bd479db8414c1353b1e0de.provisioning-net-router-interface]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645779 | hotstack | 2026-02-14 15:27:21Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-trunk0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645783 | hotstack | 2026-02-14 15:27:22Z [hs-e16e435992bd479db8414c1353b1e0de.controller-floating-ip]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645788 | hotstack | 2026-02-14 15:27:23Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645792 | hotstack | 2026-02-14 15:27:23Z [hs-e16e435992bd479db8414c1353b1e0de.controller-floating-ip]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645798 | hotstack | 2026-02-14 15:27:23Z [hs-e16e435992bd479db8414c1353b1e0de.machine-net-router-interface]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645804 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.provisioning-net-router-interface]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645810 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645816 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller-init]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645822 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller-init]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645827 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller-runcmd]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645831 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller-write-files]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645836 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller_users]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645841 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller-runcmd]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645845 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller-write-files]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645850 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.controller_users]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645854 | hotstack | 2026-02-14 15:27:24Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-trunk0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645858 | hotstack | 2026-02-14 15:27:26Z [hs-e16e435992bd479db8414c1353b1e0de.machine-net-router-interface]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645863 | hotstack | 2026-02-14 15:27:26Z [hs-e16e435992bd479db8414c1353b1e0de.router]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645867 | hotstack | 2026-02-14 15:27:28Z [hs-e16e435992bd479db8414c1353b1e0de.router]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645872 | hotstack | 2026-02-14 15:27:30Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645876 | hotstack | 2026-02-14 15:27:30Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-provisioning-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645881 | hotstack | 2026-02-14 15:27:31Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-provisioning-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645885 | hotstack | 2026-02-14 15:27:33Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-trunk0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645900 | hotstack | 2026-02-14 15:27:33Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-tenant-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645906 | hotstack | 2026-02-14 15:27:33Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-ctlplane-trunk-parent-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645915 | hotstack | 2026-02-14 15:27:33Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-internal-api-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645921 | hotstack | 2026-02-14 15:27:33Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-storage-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:46.645926 | hotstack | 2026-02-14 15:27:34Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-ctlplane-trunk-parent-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645932 | hotstack | 2026-02-14 15:27:34Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-tenant-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:46.645946 | hotstack | 2026-02-14 15:27:34Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-internal-api-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736209 | hotstack | 2026-02-14 15:27:34Z [hs-e16e435992bd479db8414c1353b1e0de.bmh0-storage-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736338 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736344 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0-lvms-vol0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736363 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0-machine-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736368 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0-cinder-vol2]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736373 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0-trunk0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736378 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0-cinder-vol0]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736383 | hotstack | 2026-02-14 15:27:35Z [hs-e16e435992bd479db8414c1353b1e0de.master0-cinder-vol1]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736389 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-machine-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736394 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.extra-dhcp-opts-value]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736398 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.extra-dhcp-opts-value]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736417 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-trunk0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736422 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.controller-machine-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736427 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-ctlplane-trunk-parent-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736432 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-tenant-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736440 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-internal-api-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736445 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-storage-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736450 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-lvms-vol0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736455 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-cinder-vol2]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736460 | hotstack | 2026-02-14 15:27:36Z [hs-e16e435992bd479db8414c1353b1e0de.master0-cinder-vol0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736464 | hotstack | 2026-02-14 15:27:37Z [hs-e16e435992bd479db8414c1353b1e0de.master0-cinder-vol1]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736469 | hotstack | 2026-02-14 15:27:37Z [hs-e16e435992bd479db8414c1353b1e0de.master0-internal-api-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736488 | hotstack | 2026-02-14 15:27:37Z [hs-e16e435992bd479db8414c1353b1e0de.master0-storage-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736492 | hotstack | 2026-02-14 15:27:37Z [hs-e16e435992bd479db8414c1353b1e0de.controller-machine-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736497 | hotstack | 2026-02-14 15:27:38Z [hs-e16e435992bd479db8414c1353b1e0de.master0-ctlplane-trunk-parent-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736502 | hotstack | 2026-02-14 15:27:38Z [hs-e16e435992bd479db8414c1353b1e0de.master0-tenant-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736507 | hotstack | 2026-02-14 15:27:40Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736518 | hotstack | 2026-02-14 15:27:41Z [hs-e16e435992bd479db8414c1353b1e0de.machine-subnet]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736523 | hotstack | 2026-02-14 15:27:41Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-provisioning-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736527 | hotstack | 2026-02-14 15:27:41Z [hs-e16e435992bd479db8414c1353b1e0de.machine-subnet]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736532 | hotstack | 2026-02-14 15:27:41Z [hs-e16e435992bd479db8414c1353b1e0de.machine-net]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736536 | hotstack | 2026-02-14 15:27:41Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-provisioning-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.736541 | hotstack | 2026-02-14 15:27:41Z [hs-e16e435992bd479db8414c1353b1e0de.provisioning-subnet]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.736565 | hotstack | 2026-02-14 15:27:42Z [hs-e16e435992bd479db8414c1353b1e0de.provisioning-subnet]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.821881 | hotstack | 2026-02-14 15:27:42Z [hs-e16e435992bd479db8414c1353b1e0de.provisioning-net]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822102 | hotstack | 2026-02-14 15:27:42Z [hs-e16e435992bd479db8414c1353b1e0de.machine-net]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822111 | hotstack | 2026-02-14 15:27:43Z [hs-e16e435992bd479db8414c1353b1e0de.provisioning-net]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822116 | hotstack | 2026-02-14 15:27:46Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-trunk0]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822135 | hotstack | 2026-02-14 15:27:46Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-internal-api-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822142 | hotstack | 2026-02-14 15:27:46Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-ctlplane-trunk-parent-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822147 | hotstack | 2026-02-14 15:27:46Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-tenant-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822152 | hotstack | 2026-02-14 15:27:46Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-storage-port]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822156 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-internal-api-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822161 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-ctlplane-trunk-parent-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822166 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-tenant-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822170 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.ctlplane-subnet]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822175 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.internal-api-subnet]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822179 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.bmh1-storage-port]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822184 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.tenant-subnet]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822188 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.storage-subnet]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822193 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.tenant-subnet]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822221 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.internal-api-subnet]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822225 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.tenant-net]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822230 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.storage-subnet]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822234 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.internal-api-net]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822239 | hotstack | 2026-02-14 15:27:47Z [hs-e16e435992bd479db8414c1353b1e0de.storage-net]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822243 | hotstack | 2026-02-14 15:27:48Z [hs-e16e435992bd479db8414c1353b1e0de.ctlplane-subnet]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822248 | hotstack | 2026-02-14 15:27:48Z [hs-e16e435992bd479db8414c1353b1e0de.ctlplane-net]: DELETE_IN_PROGRESS  state changed
2026-02-14 10:27:51.822252 | hotstack | 2026-02-14 15:27:48Z [hs-e16e435992bd479db8414c1353b1e0de.storage-net]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822257 | hotstack | 2026-02-14 15:27:48Z [hs-e16e435992bd479db8414c1353b1e0de.internal-api-net]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822261 | hotstack | 2026-02-14 15:27:48Z [hs-e16e435992bd479db8414c1353b1e0de.tenant-net]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822266 | hotstack | 2026-02-14 15:27:49Z [hs-e16e435992bd479db8414c1353b1e0de.ctlplane-net]: DELETE_COMPLETE  state changed
2026-02-14 10:27:51.822271 | hotstack | 2026-02-14 15:27:49Z [hs-e16e435992bd479db8414c1353b1e0de]: DELETE_COMPLETE  Stack DELETE completed successfully
2026-02-14 15:27:52.204803 | hotstack | changed
2026-02-14 15:27:52.217869 | 
2026-02-14 15:27:52.217996 | TASK [Print message that stack failed to delete]
2026-02-14 15:27:52.244815 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:52.258993 | 
2026-02-14 15:27:52.259292 | TASK [Print message that stack deleted successfully]
2026-02-14 15:27:52.308725 | hotstack | ok: Stack deleted successfully.
2026-02-14 15:27:52.319307 | 
2026-02-14 15:27:52.319439 | TASK [Delete clouds.yaml]
2026-02-14 15:27:52.613442 | hotstack | Output suppressed because no_log was given
2026-02-14 15:27:52.653552 | 
2026-02-14 15:27:52.653666 | PLAY RECAP
2026-02-14 15:27:52.653731 | hotstack | ok: 20 changed: 9 unreachable: 0 failed: 0 skipped: 4 rescued: 2 ignored: 1
2026-02-14 15:27:52.653764 | 
2026-02-14 15:27:52.760405 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/hotstack/post.yaml@master]
2026-02-14 15:27:52.763649 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/base-minimal/post-ssh.yaml@master]
2026-02-14 15:27:53.373598 | 
2026-02-14 15:27:53.373717 | PLAY [all]
2026-02-14 15:27:53.392622 | 
2026-02-14 15:27:53.392698 | TASK [include_role : fetch-output]
2026-02-14 15:27:53.433680 | hotstack | ok
2026-02-14 15:27:53.451035 | 
2026-02-14 15:27:53.451151 | TASK [fetch-output : Set log path for multiple nodes]
2026-02-14 15:27:53.516514 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:53.532061 | 
2026-02-14 15:27:53.532213 | TASK [fetch-output : Set log path for single node]
2026-02-14 15:27:53.579863 | hotstack | ok
2026-02-14 15:27:53.591406 | 
2026-02-14 15:27:53.591591 | LOOP [fetch-output : Ensure local output dirs]
2026-02-14 15:27:54.012516 | hotstack -> localhost | ok: "/var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/logs"
2026-02-14 15:27:54.238936 | hotstack -> localhost | changed: "/var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/artifacts"
2026-02-14 15:27:54.494596 | hotstack -> localhost | changed: "/var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/docs"
2026-02-14 15:27:54.514615 | 
2026-02-14 15:27:54.514783 | LOOP [fetch-output : Collect logs, artifacts and docs]
2026-02-14 15:27:55.196217 | hotstack | changed:
2026-02-14 15:27:55.196402 | hotstack | .d..t...... ./
2026-02-14 15:27:55.196433 | hotstack | cd+++++++++ hotlogs/
2026-02-14 15:27:55.196454 | hotstack | cd+++++++++ hotlogs/data/
2026-02-14 15:27:55.196473 | hotstack | >f+++++++++ hotlogs/data/ironic_nodes.yaml
2026-02-14 15:27:55.196492 | hotstack | cd+++++++++ hotlogs/manifests/
2026-02-14 15:27:55.196510 | hotstack | cd+++++++++ hotlogs/manifests/sushy_emulator_manifests/
2026-02-14 15:27:55.196527 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/config_map.yaml.applied
2026-02-14 15:27:55.196544 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/config_map.yaml.applied.log
2026-02-14 15:27:55.196562 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/deployment.yaml.applied
2026-02-14 15:27:55.196580 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/deployment.yaml.applied.log
2026-02-14 15:27:55.196596 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/namespace.yaml.applied
2026-02-14 15:27:55.196613 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/namespace.yaml.applied.log
2026-02-14 15:27:55.196629 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/route.yaml.20260214_152655.failed
2026-02-14 15:27:55.196646 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/route.yaml.20260214_152655.log
2026-02-14 15:27:55.196663 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/service.yaml.applied
2026-02-14 15:27:55.196682 | hotstack | >f+++++++++ hotlogs/manifests/sushy_emulator_manifests/service.yaml.applied.log
2026-02-14 15:27:55.196699 | hotstack | cd+++++++++ hotlogs/ocp_cluster/
2026-02-14 15:27:55.196716 | hotstack | >f+++++++++ hotlogs/ocp_cluster/openshift_install.log
2026-02-14 15:27:55.196733 | hotstack | cd+++++++++ hotlogs/ocp_cluster/butane/
2026-02-14 15:27:55.196750 | hotstack | >f+++++++++ hotlogs/ocp_cluster/butane/90-master-disable-netifnames.bu
2026-02-14 15:27:55.196772 | hotstack | >f+++++++++ hotlogs/ocp_cluster/butane/90-master-enable-iscsi.bu
2026-02-14 15:27:55.196789 | hotstack | >f+++++++++ hotlogs/ocp_cluster/butane/91-master-enable-multipath.bu
2026-02-14 15:27:55.196806 | hotstack | >f+++++++++ hotlogs/ocp_cluster/butane/92-master-lv-cinder-volumes.bu
2026-02-14 15:27:55.196822 | hotstack | cd+++++++++ hotlogs/ocp_cluster/config-assets/
2026-02-14 15:27:55.196840 | hotstack | >f+++++++++ hotlogs/ocp_cluster/config-assets/95-etcd_config.yaml
2026-02-14 15:27:55.196856 | hotstack | >f+++++++++ hotlogs/ocp_cluster/config-assets/ovn_k8s_config.yaml
2026-02-14 15:27:55.196873 | hotstack | cd+++++++++ hotlogs/ocp_cluster/machine-configs/
2026-02-14 15:27:55.196889 | hotstack | >f+++++++++ hotlogs/ocp_cluster/machine-configs/90-master-disable-netifnames.yaml
2026-02-14 15:27:55.196905 | hotstack | >f+++++++++ hotlogs/ocp_cluster/machine-configs/90-master-enable-iscsi.yaml
2026-02-14 15:27:55.196921 | hotstack | >f+++++++++ hotlogs/ocp_cluster/machine-configs/91-master-enable-multipath.yaml
2026-02-14 15:27:55.196937 | hotstack | >f+++++++++ hotlogs/ocp_cluster/machine-configs/92-master-lv-cinder-volumes.yaml
2026-02-14 15:27:55.196970 | hotstack | changed: All items complete
2026-02-14 15:27:55.196990 | 
2026-02-14 15:27:55.680417 | hotstack | changed: .d..t...... ./
2026-02-14 15:27:56.177698 | hotstack | changed: .d..t...... ./
2026-02-14 15:27:56.203908 | 
2026-02-14 15:27:56.204129 | TASK [include_role : fetch-output-openshift]
2026-02-14 15:27:56.229202 | hotstack | skipping: Conditional result was False
2026-02-14 15:27:56.237777 | 
2026-02-14 15:27:56.237869 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir]
2026-02-14 15:27:56.635191 | hotstack -> localhost | ok: Item: artifacts Runtime: 0:00:00.008512
2026-02-14 15:27:56.867534 | hotstack -> localhost | ok: Item: docs Runtime: 0:00:00.013446
2026-02-14 15:27:56.896108 | 
2026-02-14 15:27:56.896191 | PLAY [all]
2026-02-14 15:27:56.911518 | 
2026-02-14 15:27:56.911583 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes]
2026-02-14 15:27:57.576143 | hotstack | changed
2026-02-14 15:27:57.610224 | 
2026-02-14 15:27:57.610367 | PLAY RECAP
2026-02-14 15:27:57.610424 | hotstack | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 2 rescued: 0 ignored: 0
2026-02-14 15:27:57.610453 | 
2026-02-14 15:27:57.720261 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/base-minimal/post-ssh.yaml@master]
2026-02-14 15:27:57.728850 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/base-minimal/post-logs.yaml@master]
2026-02-14 15:27:58.374595 | 
2026-02-14 15:27:58.374708 | PLAY [localhost]
2026-02-14 15:27:58.392146 | 
2026-02-14 15:27:58.392235 | TASK [generate-zuul-manifest : Generate Zuul manifest]
2026-02-14 15:27:58.822926 | localhost | changed
2026-02-14 15:27:58.833117 | 
2026-02-14 15:27:58.833282 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul]
2026-02-14 15:27:58.870087 | localhost | ok
2026-02-14 15:27:58.888070 | 
2026-02-14 15:27:58.888222 | TASK [add-fileserver : Create SSH private key tempfile]
2026-02-14 15:27:59.342766 | localhost | changed
2026-02-14 15:27:59.353043 | 
2026-02-14 15:27:59.353193 | TASK [add-fileserver : Create SSH private key from secret]
2026-02-14 15:28:00.163320 | localhost | changed
2026-02-14 15:28:00.174196 | 
2026-02-14 15:28:00.174371 | TASK [add-fileserver : Add fileserver ssh key]
2026-02-14 15:28:00.593062 | localhost | Identity added: /var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.l5yve2m6 (/var/lib/zuul/builds/e16e435992bd479db8414c1353b1e0de/work/tmp/ansible.l5yve2m6)
2026-02-14 15:28:00.593241 | localhost | ok: Runtime: 0:00:00.012443
2026-02-14 15:28:00.597492 | 
2026-02-14 15:28:00.597559 | TASK [add-fileserver : Remove SSH private key from disk]
2026-02-14 15:28:00.829300 | localhost | ok: Runtime: 0:00:00.005300
2026-02-14 15:28:00.833950 | 
2026-02-14 15:28:00.834047 | TASK [add-fileserver : Add fileserver to inventory]
2026-02-14 15:28:00.889389 | localhost | changed
2026-02-14 15:28:00.900429 | 
2026-02-14 15:28:00.900588 | TASK [add-fileserver : Add fileserver server to known hosts]
2026-02-14 15:28:01.277132 | localhost | changed
2026-02-14 15:28:01.298035 | 
2026-02-14 15:28:01.298126 | PLAY [localhost]
2026-02-14 15:28:01.335343 | 
2026-02-14 15:28:01.335457 | TASK [Generate bulk log download script]
2026-02-14 15:28:01.364488 | localhost | ok
2026-02-14 15:28:01.377576 | 
2026-02-14 15:28:01.377683 | TASK [local-log-download : Check API endpoint is defined]
2026-02-14 15:28:01.426298 | localhost | ok: All assertions passed
2026-02-14 15:28:01.430982 | 
2026-02-14 15:28:01.431074 | TASK [local-log-download : Create download script]
2026-02-14 15:28:01.901342 | localhost -> localhost | changed
2026-02-14 15:28:01.911254 | 
2026-02-14 15:28:01.911347 | TASK [Register quick-download link]
2026-02-14 15:28:01.940011 | localhost | ok
2026-02-14 15:28:01.956550 | 
2026-02-14 15:28:01.956644 | PLAY [logserver.rdoproject.org]
2026-02-14 15:28:01.966567 | 
2026-02-14 15:28:01.966633 | TASK [Set zuul-log-path fact]
2026-02-14 15:28:01.982980 | logserver.rdoproject.org | ok
2026-02-14 15:28:01.992449 | 
2026-02-14 15:28:01.992525 | TASK [set-zuul-log-path-fact : Set log path for a build]
2026-02-14 15:28:02.018799 | logserver.rdoproject.org | ok
2026-02-14 15:28:02.024118 | 
2026-02-14 15:28:02.024184 | TASK [upload-logs : Create log directories]
2026-02-14 15:28:03.033589 | logserver.rdoproject.org | changed
2026-02-14 15:28:03.041419 | 
2026-02-14 15:28:03.041600 | TASK [upload-logs : Ensure logs are readable before uploading]
2026-02-14 15:28:03.397278 | logserver.rdoproject.org -> localhost | ok: Runtime: 0:00:00.010077
2026-02-14 15:28:03.403591 | 
2026-02-14 15:28:03.403697 | TASK [upload-logs : Upload logs to log server]
2026-02-14 15:28:04.159271 | logserver.rdoproject.org | Output suppressed because no_log was given
2026-02-14 15:28:04.165964 | 
2026-02-14 15:28:04.166150 | LOOP [upload-logs : Compress console log and json output]
2026-02-14 15:28:04.227739 | logserver.rdoproject.org | skipping: Conditional result was False
2026-02-14 15:28:04.248168 | logserver.rdoproject.org | skipping: Conditional result was False
2026-02-14 15:28:04.262144 | 
2026-02-14 15:28:04.262347 | LOOP [upload-logs : Upload compressed console log and json output]
2026-02-14 15:28:04.323555 | logserver.rdoproject.org | skipping: Conditional result was False
2026-02-14 15:28:04.324008 | 
2026-02-14 15:28:04.328220 | logserver.rdoproject.org | skipping: Conditional result was False
2026-02-14 15:28:04.340220 | 
2026-02-14 15:28:04.340405 | LOOP [upload-logs : Upload console log and json output]