2026-01-21 10:51:22.175159 | Job console starting... 2026-01-21 10:51:22.190371 | Updating repositories 2026-01-21 10:51:22.256745 | Preparing job workspace 2026-01-21 10:51:33.244217 | Running Ansible setup... 2026-01-21 10:51:38.151158 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/base-minimal/pre.yaml@master] 2026-01-21 10:51:38.760473 | 2026-01-21 10:51:38.760579 | PLAY [localhost] 2026-01-21 10:51:38.769061 | 2026-01-21 10:51:38.769131 | TASK [Gathering Facts] 2026-01-21 10:51:39.650316 | localhost | ok 2026-01-21 10:51:39.668591 | 2026-01-21 10:51:39.668725 | TASK [log-inventory : Ensure Zuul Ansible directory exists] 2026-01-21 10:51:40.100959 | localhost -> localhost | changed 2026-01-21 10:51:40.106883 | 2026-01-21 10:51:40.106954 | TASK [log-inventory : Copy ansible inventory to logs dir] 2026-01-21 10:51:40.930054 | localhost -> localhost | changed 2026-01-21 10:51:40.938509 | 2026-01-21 10:51:40.938586 | TASK [Setup log path fact] 2026-01-21 10:51:40.956935 | localhost | ok 2026-01-21 10:51:40.968623 | 2026-01-21 10:51:40.968778 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-01-21 10:51:40.996994 | localhost | ok 2026-01-21 10:51:41.005702 | 2026-01-21 10:51:41.005792 | TASK [emit-job-header : Print job information] 2026-01-21 10:51:41.045727 | # Job Information 2026-01-21 10:51:41.045912 | Ansible Version: 2.15.12 2026-01-21 10:51:41.045950 | Job: watcher-operator-kuttl 2026-01-21 10:51:41.045980 | Pipeline: github-check 2026-01-21 10:51:41.046007 | Executor: ze04.softwarefactory-project.io 2026-01-21 10:51:41.046032 | Triggered by: https://github.com/openstack-k8s-operators/watcher-operator/pull/320 2026-01-21 10:51:41.046060 | Log URL (when completed): https://logserver.rdoproject.org/4d9/rdoproject.org/4d9066f27a1f41f7af2512c243527c41/ 2026-01-21 10:51:41.046087 | Event ID: d73496b0-f6a5-11f0-975b-416b72195d48 2026-01-21 10:51:41.051569 | 2026-01-21 10:51:41.051663 | LOOP [emit-job-header : Print node information] 2026-01-21 10:51:41.160152 | localhost | ok: 2026-01-21 10:51:41.160535 | localhost | # Node Information 2026-01-21 10:51:41.160597 | localhost | Inventory Hostname: controller 2026-01-21 10:51:41.160647 | localhost | Hostname: np0005590255 2026-01-21 10:51:41.160749 | localhost | Username: zuul 2026-01-21 10:51:41.160798 | localhost | Distro: CentOS 9 2026-01-21 10:51:41.160837 | localhost | Provider: vexxhost-nodepool-tripleo 2026-01-21 10:51:41.160874 | localhost | Region: RegionOne 2026-01-21 10:51:41.160910 | localhost | Label: cloud-centos-9-stream-tripleo-vexxhost-medium 2026-01-21 10:51:41.160956 | localhost | Product Name: OpenStack Nova 2026-01-21 10:51:41.160994 | localhost | Interface IP: 38.102.83.224 2026-01-21 10:51:41.254117 | localhost | ok: 2026-01-21 10:51:41.254381 | localhost | # Node Information 2026-01-21 10:51:41.254446 | localhost | Inventory Hostname: crc 2026-01-21 10:51:41.254491 | localhost | Hostname: crc 2026-01-21 10:51:41.254531 | localhost | Username: core 2026-01-21 10:51:41.254567 | localhost | Distro: RedHat 4.18 2026-01-21 10:51:41.254603 | localhost | Provider: vexxhost-nodepool-tripleo 2026-01-21 10:51:41.254638 | localhost | Region: RegionOne 2026-01-21 10:51:41.254674 | localhost | Label: crc-cloud-ocp-4-18-1-3xl 2026-01-21 10:51:41.254759 | localhost | Product Name: OpenStack Nova 2026-01-21 10:51:41.254796 | localhost | Interface IP: 38.102.83.113 2026-01-21 10:51:41.290955 | 2026-01-21 10:51:41.291051 | PLAY [all] 2026-01-21 10:51:41.301115 | 2026-01-21 10:51:41.301201 | TASK [Gather network facts] 2026-01-21 10:51:41.773997 | controller | ok 2026-01-21 10:51:41.927566 | crc | ok 2026-01-21 10:51:41.955311 | 2026-01-21 10:51:41.955395 | TASK [include_role : start-zuul-console] 2026-01-21 10:51:41.976219 | controller | ok 2026-01-21 10:51:41.995259 | crc | ok 2026-01-21 10:51:42.014137 | 2026-01-21 10:51:42.014234 | TASK [start-zuul-console : Start zuul_console daemon.] 2026-01-21 10:51:42.404394 | controller | ok 2026-01-21 10:51:42.433265 | crc | ok 2026-01-21 10:51:42.449159 | 2026-01-21 10:51:42.449242 | TASK [include_role : add-build-sshkey] 2026-01-21 10:51:42.482310 | controller | ok 2026-01-21 10:51:42.500025 | crc | ok 2026-01-21 10:51:42.526059 | 2026-01-21 10:51:42.526185 | TASK [add-build-sshkey : Check to see if ssh key was already created for this build] 2026-01-21 10:51:42.794000 | controller -> localhost | ok 2026-01-21 10:51:42.802507 | 2026-01-21 10:51:42.802600 | TASK [add-build-sshkey : Create a new key in workspace based on build UUID] 2026-01-21 10:51:42.832172 | controller | ok 2026-01-21 10:51:42.843323 | crc | ok 2026-01-21 10:51:42.891557 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-01-21 10:51:42.891656 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/create-key-and-replace.yaml 2026-01-21 10:51:42.897812 | 2026-01-21 10:51:42.897873 | TASK [add-build-sshkey : Create Temp SSH key] 2026-01-21 10:51:43.588640 | controller -> localhost | Generating public/private rsa key pair. 2026-01-21 10:51:43.589058 | controller -> localhost | Your identification has been saved in /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/4d9066f27a1f41f7af2512c243527c41_id_rsa. 2026-01-21 10:51:43.589134 | controller -> localhost | Your public key has been saved in /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/4d9066f27a1f41f7af2512c243527c41_id_rsa.pub. 2026-01-21 10:51:43.589188 | controller -> localhost | The key fingerprint is: 2026-01-21 10:51:43.589235 | controller -> localhost | SHA256:IJejFBERQsNpHAP+i6TyRMYziPCG5n0THURwQmDt+rU zuul-build-sshkey 2026-01-21 10:51:43.589282 | controller -> localhost | The key's randomart image is: 2026-01-21 10:51:43.589326 | controller -> localhost | +---[RSA 3072]----+ 2026-01-21 10:51:43.589369 | controller -> localhost | |.=*=XOo+ | 2026-01-21 10:51:43.589412 | controller -> localhost | |. *+ o+. | 2026-01-21 10:51:43.589456 | controller -> localhost | |.o + =. | 2026-01-21 10:51:43.589497 | controller -> localhost | |o=.. =.o. | 2026-01-21 10:51:43.589537 | controller -> localhost | |+oO.o. .S | 2026-01-21 10:51:43.589583 | controller -> localhost | |=+o+. .. | 2026-01-21 10:51:43.589626 | controller -> localhost | |oo.o.o. . | 2026-01-21 10:51:43.589666 | controller -> localhost | |.o ...E | 2026-01-21 10:51:43.589752 | controller -> localhost | | . | 2026-01-21 10:51:43.589797 | controller -> localhost | +----[SHA256]-----+ 2026-01-21 10:51:43.589894 | controller -> localhost | ok: Runtime: 0:00:00.269017 2026-01-21 10:51:43.605062 | 2026-01-21 10:51:43.605233 | TASK [add-build-sshkey : Remote setup ssh keys (linux)] 2026-01-21 10:51:43.637943 | controller | ok 2026-01-21 10:51:43.656099 | crc | ok 2026-01-21 10:51:43.680451 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-01-21 10:51:43.680546 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_1/opendev.org/zuul/zuul-jobs/roles/add-build-sshkey/tasks/remote-linux.yaml 2026-01-21 10:51:43.696187 | 2026-01-21 10:51:43.696278 | TASK [add-build-sshkey : Remove previously added zuul-build-sshkey] 2026-01-21 10:51:43.715733 | controller | skipping: Conditional result was False 2026-01-21 10:51:43.738928 | crc | skipping: Conditional result was False 2026-01-21 10:51:43.746259 | 2026-01-21 10:51:43.746360 | TASK [add-build-sshkey : Enable access via build key on all nodes] 2026-01-21 10:51:44.227781 | controller | changed 2026-01-21 10:51:44.371763 | crc | changed 2026-01-21 10:51:44.385021 | 2026-01-21 10:51:44.385119 | TASK [add-build-sshkey : Make sure user has a .ssh] 2026-01-21 10:51:44.635220 | controller | ok 2026-01-21 10:51:44.681803 | crc | ok 2026-01-21 10:51:44.686391 | 2026-01-21 10:51:44.686451 | TASK [add-build-sshkey : Install build private key as SSH key on all nodes] 2026-01-21 10:51:45.458510 | controller | changed 2026-01-21 10:51:45.613739 | crc | changed 2026-01-21 10:51:45.626967 | 2026-01-21 10:51:45.627140 | TASK [add-build-sshkey : Install build public key as SSH key on all nodes] 2026-01-21 10:51:46.375602 | controller | changed 2026-01-21 10:51:46.458989 | crc | changed 2026-01-21 10:51:46.468002 | 2026-01-21 10:51:46.468119 | TASK [add-build-sshkey : Remote setup ssh keys (windows)] 2026-01-21 10:51:46.498214 | controller | skipping: Conditional result was False 2026-01-21 10:51:46.509336 | crc | skipping: Conditional result was False 2026-01-21 10:51:46.516035 | 2026-01-21 10:51:46.516137 | TASK [remove-zuul-sshkey : Remove master key from local agent] 2026-01-21 10:51:46.910260 | controller -> localhost | changed 2026-01-21 10:51:46.943156 | 2026-01-21 10:51:46.944452 | TASK [add-build-sshkey : Add back temp key] 2026-01-21 10:51:47.374216 | controller -> localhost | Identity added: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/4d9066f27a1f41f7af2512c243527c41_id_rsa (zuul-build-sshkey) 2026-01-21 10:51:47.374402 | controller -> localhost | ok: Runtime: 0:00:00.008347 2026-01-21 10:51:47.379984 | 2026-01-21 10:51:47.380045 | TASK [add-build-sshkey : Verify we can still SSH to all nodes] 2026-01-21 10:51:47.757564 | crc | ok 2026-01-21 10:51:47.767322 | controller | ok 2026-01-21 10:51:47.772216 | 2026-01-21 10:51:47.772284 | TASK [add-build-sshkey : Verify we can still SSH to all nodes (windows)] 2026-01-21 10:51:47.825113 | controller | skipping: Conditional result was False 2026-01-21 10:51:47.847910 | crc | skipping: Conditional result was False 2026-01-21 10:51:47.875738 | 2026-01-21 10:51:47.875921 | TASK [include_role : validate-host] 2026-01-21 10:51:47.920238 | controller | ok 2026-01-21 10:51:47.936122 | crc | ok 2026-01-21 10:51:47.969511 | 2026-01-21 10:51:47.969637 | TASK [validate-host : Define zuul_info_dir fact] 2026-01-21 10:51:48.013624 | controller | ok 2026-01-21 10:51:48.019229 | crc | ok 2026-01-21 10:51:48.023762 | 2026-01-21 10:51:48.023824 | TASK [validate-host : Ensure Zuul Ansible directory exists] 2026-01-21 10:51:48.260848 | controller -> localhost | ok 2026-01-21 10:51:48.267765 | 2026-01-21 10:51:48.267861 | TASK [validate-host : Collect information about the host] 2026-01-21 10:51:49.005107 | controller | ok 2026-01-21 10:51:49.265156 | crc | ok 2026-01-21 10:51:49.300129 | 2026-01-21 10:51:49.300262 | TASK [validate-host : Sanitize hostname] 2026-01-21 10:51:49.348472 | controller | ok 2026-01-21 10:51:49.388015 | crc | ok 2026-01-21 10:51:49.397430 | 2026-01-21 10:51:49.397569 | TASK [validate-host : Write out all ansible variables/facts known for each host] 2026-01-21 10:51:49.789230 | controller -> localhost | changed 2026-01-21 10:51:49.841086 | crc -> localhost | changed 2026-01-21 10:51:49.846830 | 2026-01-21 10:51:49.846901 | TASK [validate-host : Collect information about zuul worker] 2026-01-21 10:51:50.251787 | controller | ok 2026-01-21 10:51:50.298578 | crc | ok 2026-01-21 10:51:50.304304 | 2026-01-21 10:51:50.304428 | TASK [validate-host : Write out all zuul information for each host] 2026-01-21 10:51:50.735513 | crc -> localhost | changed 2026-01-21 10:51:50.767287 | controller -> localhost | changed 2026-01-21 10:51:50.782441 | 2026-01-21 10:51:50.782509 | TASK [include_role : prepare-workspace-openshift] 2026-01-21 10:51:50.796984 | controller | skipping: Conditional result was False 2026-01-21 10:51:50.802662 | 2026-01-21 10:51:50.802746 | TASK [include_role : remove-zuul-sshkey] 2026-01-21 10:51:50.817077 | controller | skipping: Conditional result was False 2026-01-21 10:51:50.822854 | 2026-01-21 10:51:50.822932 | LOOP [ensure-output-dirs : Empty Zuul Output directories by removing them] 2026-01-21 10:51:51.098548 | crc | ok: "logs" 2026-01-21 10:51:51.099200 | crc | ok: All items complete 2026-01-21 10:51:51.099275 | 2026-01-21 10:51:51.110189 | controller | ok: "logs" 2026-01-21 10:51:51.354126 | crc | ok: "artifacts" 2026-01-21 10:51:51.364874 | controller | ok: "artifacts" 2026-01-21 10:51:51.582524 | controller | ok: "docs" 2026-01-21 10:51:51.613354 | crc | ok: "docs" 2026-01-21 10:51:51.619608 | 2026-01-21 10:51:51.619672 | LOOP [ensure-output-dirs : Ensure Zuul Output directories exist] 2026-01-21 10:51:51.897540 | controller | changed: "logs" 2026-01-21 10:51:51.897801 | controller | ok: All items complete 2026-01-21 10:51:51.897835 | 2026-01-21 10:51:51.960408 | crc | changed: "logs" 2026-01-21 10:51:52.129908 | controller | changed: "artifacts" 2026-01-21 10:51:52.213318 | crc | changed: "artifacts" 2026-01-21 10:51:52.355596 | controller | changed: "docs" 2026-01-21 10:51:52.453419 | crc | changed: "docs" 2026-01-21 10:51:52.573390 | 2026-01-21 10:51:52.573527 | PLAY RECAP 2026-01-21 10:51:52.573570 | controller | ok: 22 changed: 9 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-01-21 10:51:52.573596 | crc | ok: 17 changed: 6 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-01-21 10:51:52.573618 | localhost | ok: 6 changed: 2 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-01-21 10:51:52.573636 | 2026-01-21 10:51:52.691148 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/base-minimal/pre.yaml@master] 2026-01-21 10:51:52.696153 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/ci-framework-rdo-base/pre.yaml@master] 2026-01-21 10:51:53.366008 | 2026-01-21 10:51:53.366128 | PLAY [all] 2026-01-21 10:51:53.400541 | 2026-01-21 10:51:53.400693 | TASK [mirror-info-fork : Set mirror_fqdn if not defined] 2026-01-21 10:51:53.504627 | crc | ok 2026-01-21 10:51:53.510166 | controller | ok 2026-01-21 10:51:53.523951 | 2026-01-21 10:51:53.524088 | TASK [mirror-info-fork : Create /etc/ci] 2026-01-21 10:51:53.960946 | controller | changed 2026-01-21 10:51:54.005320 | crc | changed 2026-01-21 10:51:54.013124 | 2026-01-21 10:51:54.013188 | TASK [mirror-info-fork : Install ci_mirror script] 2026-01-21 10:51:55.046648 | controller | changed 2026-01-21 10:51:55.078965 | crc | changed 2026-01-21 10:51:55.102668 | 2026-01-21 10:51:55.102799 | LOOP [add-authorized-keys : Enable access via build key on all nodes] 2026-01-21 10:51:55.534473 | controller | changed: 2026-01-21 10:51:55.534787 | controller | { 2026-01-21 10:51:55.534833 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA4Z/c9osaGGtU6X8fgELwfj/yayRurfcKA0HMFfdpPxev2dbwljysMuzoVp4OZmW1gvGtyYPSNRvnzgsaabPNKNo2ym5NToCP6UM+KSe93aln4BcM/24mXChYAbXJQ5Bqq/pIzsGs/pKetQN+vwvMxLOwTvpcsCJBXaa981RKML6xj9l/UZ7IIq1HSEKMvPLxZMWdu0Ut8DkCd5F4nOw9Wgml2uYpDCj5LLCrQQ9ChdOMz8hz6SighhNlRpPkvPaet3OXxr/ytFMu7j7vv06CaEnuMMiY2aTWN1Imin9eHAylIqFHta/3gFfQSWt9jXM7owkBLKL7ATzhaAn+fjNupw== arxcruz@redhat.com" 2026-01-21 10:51:55.534859 | controller | } 2026-01-21 10:51:55.627954 | crc | changed: 2026-01-21 10:51:55.628066 | crc | { 2026-01-21 10:51:55.628095 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA4Z/c9osaGGtU6X8fgELwfj/yayRurfcKA0HMFfdpPxev2dbwljysMuzoVp4OZmW1gvGtyYPSNRvnzgsaabPNKNo2ym5NToCP6UM+KSe93aln4BcM/24mXChYAbXJQ5Bqq/pIzsGs/pKetQN+vwvMxLOwTvpcsCJBXaa981RKML6xj9l/UZ7IIq1HSEKMvPLxZMWdu0Ut8DkCd5F4nOw9Wgml2uYpDCj5LLCrQQ9ChdOMz8hz6SighhNlRpPkvPaet3OXxr/ytFMu7j7vv06CaEnuMMiY2aTWN1Imin9eHAylIqFHta/3gFfQSWt9jXM7owkBLKL7ATzhaAn+fjNupw== arxcruz@redhat.com" 2026-01-21 10:51:55.628115 | crc | } 2026-01-21 10:51:55.805395 | controller | changed: 2026-01-21 10:51:55.805471 | controller | { 2026-01-21 10:51:55.805494 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDS4Fn6k4deCnIlOtLWqZJyksbepjQt04j8Ed8CGx9EKkj0fKiAxiI4TadXQYPuNHMixZy4Nevjb6aDhL5Z906TfvNHKUrjrG7G26a0k8vdc61NEQ7FmcGMWRLwwc6ReDO7lFpzYKBMk4YqfWgBuGU/K6WLKiVW2cVvwIuGIaYrE1OiiX0iVUUk7KApXlDJMXn7qjSYynfO4mF629NIp8FJal38+Kv+HA+0QkE5Y2xXnzD4Lar5+keymiCHRntPppXHeLIRzbt0gxC7v3L72hpQ3BTBEzwHpeS8KY+SX1y5lRMN45thCHfJqGmARJREDjBvWG8JXOPmVIKQtZmVcD5b mandreou@redhat.com" 2026-01-21 10:51:55.805514 | controller | } 2026-01-21 10:51:55.986203 | crc | changed: 2026-01-21 10:51:55.986288 | crc | { 2026-01-21 10:51:55.986311 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDS4Fn6k4deCnIlOtLWqZJyksbepjQt04j8Ed8CGx9EKkj0fKiAxiI4TadXQYPuNHMixZy4Nevjb6aDhL5Z906TfvNHKUrjrG7G26a0k8vdc61NEQ7FmcGMWRLwwc6ReDO7lFpzYKBMk4YqfWgBuGU/K6WLKiVW2cVvwIuGIaYrE1OiiX0iVUUk7KApXlDJMXn7qjSYynfO4mF629NIp8FJal38+Kv+HA+0QkE5Y2xXnzD4Lar5+keymiCHRntPppXHeLIRzbt0gxC7v3L72hpQ3BTBEzwHpeS8KY+SX1y5lRMN45thCHfJqGmARJREDjBvWG8JXOPmVIKQtZmVcD5b mandreou@redhat.com" 2026-01-21 10:51:55.986339 | crc | } 2026-01-21 10:51:56.061699 | controller | changed: 2026-01-21 10:51:56.061783 | controller | { 2026-01-21 10:51:56.061810 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9MiLfy30deHA7xPOAlew5qUq3UP2gmRMYJi8PtkjFB20/DKeWwWNnkZPqP9AayruRoo51SIiVg870gbZE2jYl+Ncx/FYDe56JeC3ySZsXoAVkC9bP7gkOGqOmJjirvAgPMI7bogVz8i+66Q4Ar7OKTp3762G4IuWPPEg4ce4Y7lx9qWocZapHYq4cYKMxrOZ7SEbFSATBbe2bPZAPKTw8do/Eny+Hq/LkHFhIeyra6cqTFQYShr+zPln0Cr+ro/pDX3bB+1ubFgTpjpkkkQsLhDfR6cCdCWM2lgnS3BTtYj5Ct9/JRPR5YOphqZz+uB+OEu2IL68hmU9vNTth1KeX rlandy@redhat.com" 2026-01-21 10:51:56.061830 | controller | } 2026-01-21 10:51:56.341353 | crc | changed: 2026-01-21 10:51:56.341480 | crc | { 2026-01-21 10:51:56.341515 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9MiLfy30deHA7xPOAlew5qUq3UP2gmRMYJi8PtkjFB20/DKeWwWNnkZPqP9AayruRoo51SIiVg870gbZE2jYl+Ncx/FYDe56JeC3ySZsXoAVkC9bP7gkOGqOmJjirvAgPMI7bogVz8i+66Q4Ar7OKTp3762G4IuWPPEg4ce4Y7lx9qWocZapHYq4cYKMxrOZ7SEbFSATBbe2bPZAPKTw8do/Eny+Hq/LkHFhIeyra6cqTFQYShr+zPln0Cr+ro/pDX3bB+1ubFgTpjpkkkQsLhDfR6cCdCWM2lgnS3BTtYj5Ct9/JRPR5YOphqZz+uB+OEu2IL68hmU9vNTth1KeX rlandy@redhat.com" 2026-01-21 10:51:56.341544 | crc | } 2026-01-21 10:51:56.350026 | controller | changed: 2026-01-21 10:51:56.350109 | controller | { 2026-01-21 10:51:56.350132 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFCbgz8gdERiJlk2IKOtkjQxEXejrio6ZYMJAVJYpOIp raukadah@gmail.com" 2026-01-21 10:51:56.350152 | controller | } 2026-01-21 10:51:56.602996 | controller | changed: 2026-01-21 10:51:56.603079 | controller | { 2026-01-21 10:51:56.603102 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBqb3Q/9uDf4LmihQ7xeJ9gA/STIQUFPSfyyV0m8AoQi bshewale@redhat.com" 2026-01-21 10:51:56.603120 | controller | } 2026-01-21 10:51:56.701057 | crc | changed: 2026-01-21 10:51:56.701143 | crc | { 2026-01-21 10:51:56.701166 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFCbgz8gdERiJlk2IKOtkjQxEXejrio6ZYMJAVJYpOIp raukadah@gmail.com" 2026-01-21 10:51:56.701184 | crc | } 2026-01-21 10:51:56.869079 | controller | changed: 2026-01-21 10:51:56.869185 | controller | { 2026-01-21 10:51:56.869218 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0I8QqQx0Az2ysJt2JuffucLijhBqnsXKEIx5GyHwxVULROa8VtNFXUDH6ZKZavhiMcmfHB2+TBTda+lDP4FldYj06dGmzCY+IYGa+uDRdxHNGYjvCfLFcmLlzRK6fNbTcui+KlUFUdKe0fb9CRoGKyhlJD5GRkM1Dv+Yb6Bj+RNnmm1fVGYxzmrD2utvffYEb0SZGWxq2R9gefx1q/3wCGjeqvufEV+AskPhVGc5T7t9eyZ4qmslkLh1/nMuaIBFcr9AUACRajsvk6mXrAN1g3HlBf2gQlhi1UEyfbqIQvzzFtsbLDlSum/KmKjy818GzvWjERfQ0VkGzCd9bSLVL dviroel@redhat.com" 2026-01-21 10:51:56.869246 | controller | } 2026-01-21 10:51:57.108734 | crc | changed: 2026-01-21 10:51:57.108836 | crc | { 2026-01-21 10:51:57.108998 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBqb3Q/9uDf4LmihQ7xeJ9gA/STIQUFPSfyyV0m8AoQi bshewale@redhat.com" 2026-01-21 10:51:57.109029 | crc | } 2026-01-21 10:51:57.136112 | controller | changed: 2026-01-21 10:51:57.136209 | controller | { 2026-01-21 10:51:57.136232 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLOQd4ZLtkZXQGY6UwAr/06ppWQK4fDO3HaqxPk98csyOCBXsliSKK39Bso828+5srIXiW7aI6aC9P5mwi4mUZlGPfJlQbfrcGvY+b/SocuvaGK+1RrHLoJCT52LBhwgrzlXio2jeksZeein8iaTrhsPrOAs7KggIL/rB9hEiB3NaOPWhhoCP4vlW6MEMExGcqB/1FVxXFBPnLkEyW0Lk7ycVflZl2ocRxbfjZi0+tI1Wlinp8PvSQSc/WVrAcDgKjc/mB4ODPOyYy3G8FHgfMsrXSDEyjBKgLKMsdCrAUcqJQWjkqXleXSYOV4q3pzL+9umK+q/e3P/bIoSFQzmJKTU1eDfuvPXmow9F5H54fii/Da7ezlMJ+wPGHJrRAkmzvMbALy7xwswLhZMkOGNtRcPqaKYRmIBKpw3o6bCTtcNUHOtOQnzwY8JzrM2eBWJBXAANYw+9/ho80JIiwhg29CFNpVBuHbql2YxJQNrnl90guN65rYNpDxdIluweyUf8= anbanerj@kaermorhen" 2026-01-21 10:51:57.136252 | controller | } 2026-01-21 10:51:57.444317 | controller | changed: 2026-01-21 10:51:57.444472 | controller | { 2026-01-21 10:51:57.444525 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VwV8Im9kRm49lt3tM36hj4Zv27FxGo4C1Q/0jqhzFmHY7RHbmeRr8ObhwWoHjXSozKWg8FL5ER0z3hTwL0W6lez3sL7hUaCmSuZmG5Hnl3x4vTSxDI9JZ/Y65rtYiiWQo2fC5xJhU/4+0e5e/pseCm8cKRSu+SaxhO+sd6FDojA2x1BzOzKiQRDy/1zWGp/cZkxcEuB1wHI5LMzN03c67vmbu+fhZRAUO4dQkvcnj2LrhQtpa+ytvnSjr8icMDosf1OsbSffwZFyHB/hfWGAfe0eIeSA2XPraxiPknXxiPKx2MJsaUTYbsZcm3EjFdHBBMumw5rBI74zLrMRvCO9GwBEmGT4rFng1nP+yw5DB8sn2zqpOsPg1LYRwCPOUveC13P6pgsZZPh812e8v5EKnETct+5XI3dVpdw6CnNiLwAyVAF15DJvBGT/u1k0Myg/bQn+Gv9k2MSj6LvQmf6WbZu2Wgjm30z3FyCneBqTL7mLF19YXzeC0ufHz5pnO1E= dasm@fedora" 2026-01-21 10:51:57.444568 | controller | } 2026-01-21 10:51:57.499488 | crc | changed: 2026-01-21 10:51:57.499651 | crc | { 2026-01-21 10:51:57.499713 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0I8QqQx0Az2ysJt2JuffucLijhBqnsXKEIx5GyHwxVULROa8VtNFXUDH6ZKZavhiMcmfHB2+TBTda+lDP4FldYj06dGmzCY+IYGa+uDRdxHNGYjvCfLFcmLlzRK6fNbTcui+KlUFUdKe0fb9CRoGKyhlJD5GRkM1Dv+Yb6Bj+RNnmm1fVGYxzmrD2utvffYEb0SZGWxq2R9gefx1q/3wCGjeqvufEV+AskPhVGc5T7t9eyZ4qmslkLh1/nMuaIBFcr9AUACRajsvk6mXrAN1g3HlBf2gQlhi1UEyfbqIQvzzFtsbLDlSum/KmKjy818GzvWjERfQ0VkGzCd9bSLVL dviroel@redhat.com" 2026-01-21 10:51:57.499742 | crc | } 2026-01-21 10:51:57.718293 | controller | changed: 2026-01-21 10:51:57.718391 | controller | { 2026-01-21 10:51:57.718414 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHUnwjB20UKmsSed9X73eGNV5AOEFccQ3NYrRW776pEk cjeanner" 2026-01-21 10:51:57.718432 | controller | } 2026-01-21 10:51:57.894196 | crc | changed: 2026-01-21 10:51:57.894346 | crc | { 2026-01-21 10:51:57.894396 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLOQd4ZLtkZXQGY6UwAr/06ppWQK4fDO3HaqxPk98csyOCBXsliSKK39Bso828+5srIXiW7aI6aC9P5mwi4mUZlGPfJlQbfrcGvY+b/SocuvaGK+1RrHLoJCT52LBhwgrzlXio2jeksZeein8iaTrhsPrOAs7KggIL/rB9hEiB3NaOPWhhoCP4vlW6MEMExGcqB/1FVxXFBPnLkEyW0Lk7ycVflZl2ocRxbfjZi0+tI1Wlinp8PvSQSc/WVrAcDgKjc/mB4ODPOyYy3G8FHgfMsrXSDEyjBKgLKMsdCrAUcqJQWjkqXleXSYOV4q3pzL+9umK+q/e3P/bIoSFQzmJKTU1eDfuvPXmow9F5H54fii/Da7ezlMJ+wPGHJrRAkmzvMbALy7xwswLhZMkOGNtRcPqaKYRmIBKpw3o6bCTtcNUHOtOQnzwY8JzrM2eBWJBXAANYw+9/ho80JIiwhg29CFNpVBuHbql2YxJQNrnl90guN65rYNpDxdIluweyUf8= anbanerj@kaermorhen" 2026-01-21 10:51:57.894438 | crc | } 2026-01-21 10:51:58.000294 | controller | changed: 2026-01-21 10:51:58.000464 | controller | { 2026-01-21 10:51:58.000522 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDercCMGn8rW1C4P67tHgtflPdTeXlpyUJYH+6XDd2lR jgilaber@redhat.com" 2026-01-21 10:51:58.000564 | controller | } 2026-01-21 10:51:58.255718 | controller | changed: 2026-01-21 10:51:58.255860 | controller | { 2026-01-21 10:51:58.255906 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAMI6kkg9Wg0sG7jIJmyZemEBwUn1yzNpQQd3gnulOmZ adrianfuscoarnejo@gmail.com" 2026-01-21 10:51:58.255940 | controller | } 2026-01-21 10:51:58.274051 | crc | changed: 2026-01-21 10:51:58.274332 | crc | { 2026-01-21 10:51:58.274375 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VwV8Im9kRm49lt3tM36hj4Zv27FxGo4C1Q/0jqhzFmHY7RHbmeRr8ObhwWoHjXSozKWg8FL5ER0z3hTwL0W6lez3sL7hUaCmSuZmG5Hnl3x4vTSxDI9JZ/Y65rtYiiWQo2fC5xJhU/4+0e5e/pseCm8cKRSu+SaxhO+sd6FDojA2x1BzOzKiQRDy/1zWGp/cZkxcEuB1wHI5LMzN03c67vmbu+fhZRAUO4dQkvcnj2LrhQtpa+ytvnSjr8icMDosf1OsbSffwZFyHB/hfWGAfe0eIeSA2XPraxiPknXxiPKx2MJsaUTYbsZcm3EjFdHBBMumw5rBI74zLrMRvCO9GwBEmGT4rFng1nP+yw5DB8sn2zqpOsPg1LYRwCPOUveC13P6pgsZZPh812e8v5EKnETct+5XI3dVpdw6CnNiLwAyVAF15DJvBGT/u1k0Myg/bQn+Gv9k2MSj6LvQmf6WbZu2Wgjm30z3FyCneBqTL7mLF19YXzeC0ufHz5pnO1E= dasm@fedora" 2026-01-21 10:51:58.274403 | crc | } 2026-01-21 10:51:58.532753 | controller | changed: 2026-01-21 10:51:58.534814 | controller | { 2026-01-21 10:51:58.534918 | controller | "public_key": "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPijwpQu/3jhhhBZInXNOLEH57DrknPc3PLbsRvYyJIFzwYjX+WD4a7+nGnMYS42MuZk6TJcVqgnqofVx4isoD4= ramishra@redhat.com" 2026-01-21 10:51:58.534977 | controller | } 2026-01-21 10:51:58.651474 | crc | changed: 2026-01-21 10:51:58.651604 | crc | { 2026-01-21 10:51:58.651638 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHUnwjB20UKmsSed9X73eGNV5AOEFccQ3NYrRW776pEk cjeanner" 2026-01-21 10:51:58.651667 | crc | } 2026-01-21 10:51:58.780310 | controller | changed: 2026-01-21 10:51:58.780405 | controller | { 2026-01-21 10:51:58.780429 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGpU/BepK3qX0NRf5Np+dOBDqzQEefhNrw2DCZaH3uWW rebtoor@monolith" 2026-01-21 10:51:58.780449 | controller | } 2026-01-21 10:51:59.029177 | crc | changed: 2026-01-21 10:51:59.029615 | crc | { 2026-01-21 10:51:59.029664 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDercCMGn8rW1C4P67tHgtflPdTeXlpyUJYH+6XDd2lR jgilaber@redhat.com" 2026-01-21 10:51:59.029725 | crc | } 2026-01-21 10:51:59.058828 | controller | changed: 2026-01-21 10:51:59.058952 | controller | { 2026-01-21 10:51:59.058991 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDK0iKdi8jQTpQrDdLVH/AAgLVYyTXF7AQ1gjc/5uT3t ykarel@yatinkarel" 2026-01-21 10:51:59.059025 | controller | } 2026-01-21 10:51:59.373837 | controller | changed: 2026-01-21 10:51:59.373987 | controller | { 2026-01-21 10:51:59.374221 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF/V/cLotA6LZeO32VL45Hd78skuA2lJA425Sm2LlQeZ fmount@horcrux" 2026-01-21 10:51:59.374263 | controller | } 2026-01-21 10:51:59.414548 | crc | changed: 2026-01-21 10:51:59.414825 | crc | { 2026-01-21 10:51:59.414894 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAMI6kkg9Wg0sG7jIJmyZemEBwUn1yzNpQQd3gnulOmZ adrianfuscoarnejo@gmail.com" 2026-01-21 10:51:59.414938 | crc | } 2026-01-21 10:51:59.663384 | controller | changed: 2026-01-21 10:51:59.663535 | controller | { 2026-01-21 10:51:59.663586 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDa7QCjuDMVmRPo1rREbGwzYeBCYVN+Ou/3WKXZEC6Sr" 2026-01-21 10:51:59.663625 | controller | } 2026-01-21 10:51:59.841953 | crc | changed: 2026-01-21 10:51:59.842115 | crc | { 2026-01-21 10:51:59.842153 | crc | "public_key": "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPijwpQu/3jhhhBZInXNOLEH57DrknPc3PLbsRvYyJIFzwYjX+WD4a7+nGnMYS42MuZk6TJcVqgnqofVx4isoD4= ramishra@redhat.com" 2026-01-21 10:51:59.842181 | crc | } 2026-01-21 10:51:59.935262 | controller | changed: 2026-01-21 10:51:59.935357 | controller | { 2026-01-21 10:51:59.935381 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCfNtF7NvKl915TGsGGoseUb06Hj8L/S4toWf0hExeY+F00woL6NvBlJD0nDct+P5a22I4EhvoQCRQ8reaPCm1lybR3uiRIJsj+8zkVvLwby9LXzfZorlNG9ofjd00FEmB09uW/YvTl6Q9XwwwX6tInzIOv3TMqTHHGOL74ibbj8J/FJR0cFEyj0z4WQRvtkh32xAHl83gbuINryMt0sqRI+clj2381NKL55DRLQrVw0gsfqqxiHAnXg21qWmc4J+b9e9kiuAFQjcjwTVkwJCcg3xbPwC/qokYRby/Y5S40UUd7/jEARGXT7RZgpzTuDd1oZiCVrnrqJNPaMNdVv5MLeFdf1B7iIe5aa/fGouX7AO4SdKhZUdnJmCFAGvjC6S3JMZ2wAcUl+OHnssfmdj7XL50cLo27vjuzMtLAgSqi6N99m92WCF2s8J9aVzszX7Xz9OKZCeGsiVJp3/NdABKzSEAyM9xBD/5Vho894Sav+otpySHe3p6RUTgbB5Zu8VyZRZ/UtB3ueXxyo764yrc6qWIDqrehm84Xm9g+/jpIBzGPl07NUNJpdt/6Sgf9RIKXw/7XypO5yZfUcuFNGTxLfqjTNrtgLZNcjfav6sSdVXVcMPL//XNuRdKmVFaO76eV/oGMQGr1fGcCD+N+CpI7+Q+fCNB6VFWG4nZFuI/Iuw== averdagu@redhat.com" 2026-01-21 10:51:59.935401 | controller | } 2026-01-21 10:52:00.218451 | controller | changed: 2026-01-21 10:52:00.218571 | controller | { 2026-01-21 10:52:00.218610 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDq8l27xI+QlQVdS4djp9ogSoyrNE2+Ox6vKPdhSNL1J3PE5w+WCSvMz9A5gnNuH810zwbekEApbxTze/gLQJwBHA52CChfURpXrFaxY7ePXRElwKAL3mJfzBWY/c5jnNL9TCVmFJTGZkFZP3Nh+BMgZvL6xBkt3WKm6Uq18qzd9XeKcZusrA+O+uLv1fVeQnadY9RIqOCyeFYCzLWrUfTyE8x/XG0hAWIM7qpnF2cALQS2h9n4hW5ybiUN790H08wf9hFwEf5nxY9Z9dVkPFQiTSGKNBzmnCXU9skxS/xhpFjJ5duGSZdtAHe9O+nGZm9c67hxgtf8e5PDuqAdXEv2cf6e3VBAt+Bz8EKI3yosTj0oZHfwr42Yzb1l/SKy14Rggsrc9KAQlrGXan6+u2jcQqqx7l+SWmnpFiWTV9u5cWj2IgOhApOitmRBPYqk9rE2usfO0hLn/Pj/R/Nau4803e1/EikdLE7Ps95s9mX5jRDjAoUa2JwFF5RsVFyL910= ashigupt@ashigupt.remote.csb" 2026-01-21 10:52:00.218634 | controller | } 2026-01-21 10:52:00.337670 | crc | changed: 2026-01-21 10:52:00.337873 | crc | { 2026-01-21 10:52:00.337947 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGpU/BepK3qX0NRf5Np+dOBDqzQEefhNrw2DCZaH3uWW rebtoor@monolith" 2026-01-21 10:52:00.337992 | crc | } 2026-01-21 10:52:00.467207 | controller | changed: 2026-01-21 10:52:00.467373 | controller | { 2026-01-21 10:52:00.467404 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOKLl0NYKwoZ/JY5KeZU8VwRAggeOxqQJeoqp3dsAaY9" 2026-01-21 10:52:00.467426 | controller | } 2026-01-21 10:52:00.685531 | crc | changed: 2026-01-21 10:52:00.685666 | crc | { 2026-01-21 10:52:00.685712 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDK0iKdi8jQTpQrDdLVH/AAgLVYyTXF7AQ1gjc/5uT3t ykarel@yatinkarel" 2026-01-21 10:52:00.685732 | crc | } 2026-01-21 10:52:01.066604 | crc | changed: 2026-01-21 10:52:01.066735 | crc | { 2026-01-21 10:52:01.066775 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF/V/cLotA6LZeO32VL45Hd78skuA2lJA425Sm2LlQeZ fmount@horcrux" 2026-01-21 10:52:01.066805 | crc | } 2026-01-21 10:52:01.143315 | controller | changed: 2026-01-21 10:52:01.143403 | controller | { 2026-01-21 10:52:01.143428 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIASASQOH2BcOyLKuuDOdWZlPi2orcjcA8q4400T73DLH evallesp@fedora" 2026-01-21 10:52:01.143448 | controller | } 2026-01-21 10:52:01.405249 | crc | changed: 2026-01-21 10:52:01.405332 | crc | { 2026-01-21 10:52:01.405354 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDa7QCjuDMVmRPo1rREbGwzYeBCYVN+Ou/3WKXZEC6Sr" 2026-01-21 10:52:01.405373 | crc | } 2026-01-21 10:52:01.772495 | crc | changed: 2026-01-21 10:52:01.772594 | crc | { 2026-01-21 10:52:01.772626 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCfNtF7NvKl915TGsGGoseUb06Hj8L/S4toWf0hExeY+F00woL6NvBlJD0nDct+P5a22I4EhvoQCRQ8reaPCm1lybR3uiRIJsj+8zkVvLwby9LXzfZorlNG9ofjd00FEmB09uW/YvTl6Q9XwwwX6tInzIOv3TMqTHHGOL74ibbj8J/FJR0cFEyj0z4WQRvtkh32xAHl83gbuINryMt0sqRI+clj2381NKL55DRLQrVw0gsfqqxiHAnXg21qWmc4J+b9e9kiuAFQjcjwTVkwJCcg3xbPwC/qokYRby/Y5S40UUd7/jEARGXT7RZgpzTuDd1oZiCVrnrqJNPaMNdVv5MLeFdf1B7iIe5aa/fGouX7AO4SdKhZUdnJmCFAGvjC6S3JMZ2wAcUl+OHnssfmdj7XL50cLo27vjuzMtLAgSqi6N99m92WCF2s8J9aVzszX7Xz9OKZCeGsiVJp3/NdABKzSEAyM9xBD/5Vho894Sav+otpySHe3p6RUTgbB5Zu8VyZRZ/UtB3ueXxyo764yrc6qWIDqrehm84Xm9g+/jpIBzGPl07NUNJpdt/6Sgf9RIKXw/7XypO5yZfUcuFNGTxLfqjTNrtgLZNcjfav6sSdVXVcMPL//XNuRdKmVFaO76eV/oGMQGr1fGcCD+N+CpI7+Q+fCNB6VFWG4nZFuI/Iuw== averdagu@redhat.com" 2026-01-21 10:52:01.772658 | crc | } 2026-01-21 10:52:01.800112 | controller | changed: 2026-01-21 10:52:01.800219 | controller | { 2026-01-21 10:52:01.800252 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILeBWlamUph+jRKV2qrx1PGU7vWuGIt5+z9k96I8WehW amsinha@amsinha-mac" 2026-01-21 10:52:01.800279 | controller | } 2026-01-21 10:52:02.048635 | controller | changed: 2026-01-21 10:52:02.048739 | controller | { 2026-01-21 10:52:02.048764 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIANvVgvJBlK3gb1yz5uef/JqIGq4HLEmY2dYA8e37swb morenod@redhat-laptop" 2026-01-21 10:52:02.048784 | controller | } 2026-01-21 10:52:02.161748 | crc | changed: 2026-01-21 10:52:02.161841 | crc | { 2026-01-21 10:52:02.161865 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDq8l27xI+QlQVdS4djp9ogSoyrNE2+Ox6vKPdhSNL1J3PE5w+WCSvMz9A5gnNuH810zwbekEApbxTze/gLQJwBHA52CChfURpXrFaxY7ePXRElwKAL3mJfzBWY/c5jnNL9TCVmFJTGZkFZP3Nh+BMgZvL6xBkt3WKm6Uq18qzd9XeKcZusrA+O+uLv1fVeQnadY9RIqOCyeFYCzLWrUfTyE8x/XG0hAWIM7qpnF2cALQS2h9n4hW5ybiUN790H08wf9hFwEf5nxY9Z9dVkPFQiTSGKNBzmnCXU9skxS/xhpFjJ5duGSZdtAHe9O+nGZm9c67hxgtf8e5PDuqAdXEv2cf6e3VBAt+Bz8EKI3yosTj0oZHfwr42Yzb1l/SKy14Rggsrc9KAQlrGXan6+u2jcQqqx7l+SWmnpFiWTV9u5cWj2IgOhApOitmRBPYqk9rE2usfO0hLn/Pj/R/Nau4803e1/EikdLE7Ps95s9mX5jRDjAoUa2JwFF5RsVFyL910= ashigupt@ashigupt.remote.csb" 2026-01-21 10:52:02.161886 | crc | } 2026-01-21 10:52:02.315023 | controller | changed: 2026-01-21 10:52:02.315110 | controller | { 2026-01-21 10:52:02.315132 | controller | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZdI7t1cxYx65heVI24HTV4F7oQLW1zyfxHreL2TIJKxjyrUUKIFEUmTutcBlJRLNT2Eoix6x1sOw9YrchloCLcn//SGfTElr9mSc5jbjb7QXEU+zJMhtxyEJ1Po3CUGnj7ckiIXw7wcawZtrEOAQ9pH3ExYCJcEMiyNjRQZCxT3tPK+S4B95EWh5Fsrz9CkwpjNRPPH7LigCeQTM3Wc7r97utAslBUUvYceDSLA7rMgkitJE38b7rZBeYzsGQ8YYUBjTCtehqQXxCRjizbHWaaZkBU+N3zkKB6n/iCNGIO690NK7A/qb6msTijiz1PeuM8ThOsi9qXnbX5v0PoTpcFSojV7NHAQ71f0XXuS43FhZctT+Dcx44dT8Fb5vJu2cJGrk+qF8ZgJYNpRS7gPg0EG2EqjK7JMf9ULdjSu0r+KlqIAyLvtzT4eOnQipoKlb/WG5D/0ohKv7OMQ352ggfkBFIQsRXyyTCT98Ft9juqPuahi3CAQmP4H9dyE+7+Kz437PEtsxLmfm6naNmWi7Ee1DqWPwS8rEajsm4sNM4wW9gdBboJQtc0uZw0DfLj1I9r3Mc8Ol0jYtz0yNQDSzVLrGCaJlC311trU70tZ+ZkAVV6Mn8lOhSbj1cK0lvSr6ZK4dgqGl3I1eTZJJhbLNdg7UOVaiRx9543+C/p/As7w== brjackma@redhat.com" 2026-01-21 10:52:02.315160 | controller | } 2026-01-21 10:52:02.500572 | crc | changed: 2026-01-21 10:52:02.500700 | crc | { 2026-01-21 10:52:02.500781 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOKLl0NYKwoZ/JY5KeZU8VwRAggeOxqQJeoqp3dsAaY9" 2026-01-21 10:52:02.500805 | crc | } 2026-01-21 10:52:02.594366 | controller | changed: 2026-01-21 10:52:02.594451 | controller | { 2026-01-21 10:52:02.594475 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKwedoZ0TWPJX/z/4TAbO/kKcDZOQVgRH0hAqrL5UCI1 vcastell@redhat.com" 2026-01-21 10:52:02.594494 | controller | } 2026-01-21 10:52:02.860594 | controller | changed: 2026-01-21 10:52:02.860744 | controller | { 2026-01-21 10:52:02.860774 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEmv8sE8GCk6ZTPIqF0FQrttBdL3mq7rCm/IJy0xDFh7 michburk@redhat.com" 2026-01-21 10:52:02.860793 | controller | } 2026-01-21 10:52:02.886034 | crc | changed: 2026-01-21 10:52:02.886165 | crc | { 2026-01-21 10:52:02.886193 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIASASQOH2BcOyLKuuDOdWZlPi2orcjcA8q4400T73DLH evallesp@fedora" 2026-01-21 10:52:02.886214 | crc | } 2026-01-21 10:52:03.138019 | controller | changed: 2026-01-21 10:52:03.138131 | controller | { 2026-01-21 10:52:03.138164 | controller | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICy6GpGEtwevXEEn4mmLR5lmSLe23dGgAvzkB9DMNbkf rsafrono@rsafrono" 2026-01-21 10:52:03.138191 | controller | } 2026-01-21 10:52:03.274374 | crc | changed: 2026-01-21 10:52:03.274613 | crc | { 2026-01-21 10:52:03.274657 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILeBWlamUph+jRKV2qrx1PGU7vWuGIt5+z9k96I8WehW amsinha@amsinha-mac" 2026-01-21 10:52:03.274741 | crc | } 2026-01-21 10:52:03.274808 | crc | changed: All items complete 2026-01-21 10:52:03.274853 | 2026-01-21 10:52:03.672897 | crc | changed: 2026-01-21 10:52:03.673155 | crc | { 2026-01-21 10:52:03.673195 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIANvVgvJBlK3gb1yz5uef/JqIGq4HLEmY2dYA8e37swb morenod@redhat-laptop" 2026-01-21 10:52:03.673223 | crc | } 2026-01-21 10:52:04.077525 | crc | changed: 2026-01-21 10:52:04.077632 | crc | { 2026-01-21 10:52:04.077666 | crc | "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZdI7t1cxYx65heVI24HTV4F7oQLW1zyfxHreL2TIJKxjyrUUKIFEUmTutcBlJRLNT2Eoix6x1sOw9YrchloCLcn//SGfTElr9mSc5jbjb7QXEU+zJMhtxyEJ1Po3CUGnj7ckiIXw7wcawZtrEOAQ9pH3ExYCJcEMiyNjRQZCxT3tPK+S4B95EWh5Fsrz9CkwpjNRPPH7LigCeQTM3Wc7r97utAslBUUvYceDSLA7rMgkitJE38b7rZBeYzsGQ8YYUBjTCtehqQXxCRjizbHWaaZkBU+N3zkKB6n/iCNGIO690NK7A/qb6msTijiz1PeuM8ThOsi9qXnbX5v0PoTpcFSojV7NHAQ71f0XXuS43FhZctT+Dcx44dT8Fb5vJu2cJGrk+qF8ZgJYNpRS7gPg0EG2EqjK7JMf9ULdjSu0r+KlqIAyLvtzT4eOnQipoKlb/WG5D/0ohKv7OMQ352ggfkBFIQsRXyyTCT98Ft9juqPuahi3CAQmP4H9dyE+7+Kz437PEtsxLmfm6naNmWi7Ee1DqWPwS8rEajsm4sNM4wW9gdBboJQtc0uZw0DfLj1I9r3Mc8Ol0jYtz0yNQDSzVLrGCaJlC311trU70tZ+ZkAVV6Mn8lOhSbj1cK0lvSr6ZK4dgqGl3I1eTZJJhbLNdg7UOVaiRx9543+C/p/As7w== brjackma@redhat.com" 2026-01-21 10:52:04.077724 | crc | } 2026-01-21 10:52:04.456567 | crc | changed: 2026-01-21 10:52:04.456763 | crc | { 2026-01-21 10:52:04.456825 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKwedoZ0TWPJX/z/4TAbO/kKcDZOQVgRH0hAqrL5UCI1 vcastell@redhat.com" 2026-01-21 10:52:04.456874 | crc | } 2026-01-21 10:52:04.810595 | crc | changed: 2026-01-21 10:52:04.810731 | crc | { 2026-01-21 10:52:04.810766 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEmv8sE8GCk6ZTPIqF0FQrttBdL3mq7rCm/IJy0xDFh7 michburk@redhat.com" 2026-01-21 10:52:04.810802 | crc | } 2026-01-21 10:52:05.174704 | crc | changed: 2026-01-21 10:52:05.176071 | crc | { 2026-01-21 10:52:05.176149 | crc | "public_key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICy6GpGEtwevXEEn4mmLR5lmSLe23dGgAvzkB9DMNbkf rsafrono@rsafrono" 2026-01-21 10:52:05.176192 | crc | } 2026-01-21 10:52:05.234626 | 2026-01-21 10:52:05.234781 | TASK [Set timezone to UTC] 2026-01-21 10:52:05.812994 | controller | changed: executed `/bin/timedatectl set-timezone UTC` 2026-01-21 10:52:06.844985 | crc | ok 2026-01-21 10:52:06.850229 | 2026-01-21 10:52:06.850291 | TASK [Create nodepool directory] 2026-01-21 10:52:07.147375 | controller | changed 2026-01-21 10:52:07.184866 | crc | changed 2026-01-21 10:52:07.190299 | 2026-01-21 10:52:07.190359 | TASK [Create nodepool sub_nodes file] 2026-01-21 10:52:07.893904 | controller | changed 2026-01-21 10:52:07.954189 | crc | changed 2026-01-21 10:52:07.959010 | 2026-01-21 10:52:07.959071 | TASK [Create nodepool sub_nodes_private file] 2026-01-21 10:52:08.625203 | controller | changed 2026-01-21 10:52:08.684348 | crc | changed 2026-01-21 10:52:08.692672 | 2026-01-21 10:52:08.692848 | LOOP [Populate nodepool sub_nodes file] 2026-01-21 10:52:08.750063 | 2026-01-21 10:52:08.750236 | LOOP [Populate nodepool sub_nodes_private file] 2026-01-21 10:52:08.832030 | 2026-01-21 10:52:08.832291 | TASK [Create nodepool primary file] 2026-01-21 10:52:08.894975 | controller | skipping: Conditional result was False 2026-01-21 10:52:08.895561 | crc | skipping: Conditional result was False 2026-01-21 10:52:08.901103 | 2026-01-21 10:52:08.901166 | TASK [Create nodepool node_private for this node] 2026-01-21 10:52:09.740668 | crc | changed 2026-01-21 10:52:09.918631 | controller | changed 2026-01-21 10:52:09.924370 | 2026-01-21 10:52:09.924439 | LOOP [Copy ssh keys to nodepool directory] 2026-01-21 10:52:10.336857 | controller | ok: Item: id_rsa Runtime: 0:00:00.006112 2026-01-21 10:52:10.337014 | 2026-01-21 10:52:10.363827 | crc | ok: Item: id_rsa Runtime: 0:00:00.006307 2026-01-21 10:52:10.547854 | controller | ok: Item: id_rsa.pub Runtime: 0:00:00.004967 2026-01-21 10:52:10.625145 | crc | ok: Item: id_rsa.pub Runtime: 0:00:00.005342 2026-01-21 10:52:10.639020 | 2026-01-21 10:52:10.639602 | TASK [Add sudoers role for zuul-sudo-grep.sh] 2026-01-21 10:52:11.392613 | controller | changed 2026-01-21 10:52:11.461467 | crc | changed 2026-01-21 10:52:11.477897 | 2026-01-21 10:52:11.478037 | TASK [Validate sudoers config after edits] 2026-01-21 10:52:11.879181 | crc | /etc/sudoers: parsed OK 2026-01-21 10:52:11.879309 | crc | /etc/sudoers.d/coreos-sudo-group: parsed OK 2026-01-21 10:52:11.879319 | crc | /etc/sudoers.d/zuul-sudo-grep: parsed OK 2026-01-21 10:52:11.834358 | controller | /etc/sudoers: parsed OK 2026-01-21 10:52:11.834544 | controller | /etc/sudoers.d/90-cloud-init-users: parsed OK 2026-01-21 10:52:11.834609 | controller | /etc/sudoers.d/zuul: parsed OK 2026-01-21 10:52:11.834616 | controller | /etc/sudoers.d/zuul-sudo-grep: parsed OK 2026-01-21 10:52:12.073320 | controller | ok: Runtime: 0:00:00.006448 2026-01-21 10:52:12.090590 | crc | ok: Runtime: 0:00:00.010554 2026-01-21 10:52:12.104039 | 2026-01-21 10:52:12.104170 | TASK [Show the environment passed in to job shell scripts] 2026-01-21 10:52:12.478283 | crc | SHELL=/bin/bash 2026-01-21 10:52:12.478452 | crc | ZUUL_PROJECT=openstack-k8s-operators/watcher-operator 2026-01-21 10:52:12.478469 | crc | ZUUL_SHORT_PROJECT_NAME=watcher-operator 2026-01-21 10:52:12.478478 | crc | ZUUL_CHANGES=openstack-k8s-operators/watcher-operator:main:refs/changes/20/320/111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 10:52:12.478544 | crc | PWD=/var/home/core 2026-01-21 10:52:12.478553 | crc | ZUUL_PIPELINE=github-check 2026-01-21 10:52:12.478561 | crc | LOGNAME=core 2026-01-21 10:52:12.478569 | crc | XDG_SESSION_TYPE=tty 2026-01-21 10:52:12.478577 | crc | _=/usr/bin/env 2026-01-21 10:52:12.478584 | crc | MOTD_SHOWN=pam 2026-01-21 10:52:12.478590 | crc | HOME=/var/home/core 2026-01-21 10:52:12.478597 | crc | LANG=C.UTF-8 2026-01-21 10:52:12.478603 | crc | SSH_CONNECTION=38.102.83.114 38574 38.102.83.113 22 2026-01-21 10:52:12.478611 | crc | TOX_TESTENV_PASSENV=ZUUL_UUID ZUUL_PROJECT ZUUL_SHORT_PROJECT_NAME ZUUL_PIPELINE ZUUL_VOTING WORKSPACE BUILD_TIMEOUT ZUUL_BRANCH ZUUL_CHANGES ZUUL_CHANGE_IDS ZUUL_CHANGE ZUUL_PATCHSET 2026-01-21 10:52:12.478622 | crc | ZUUL_CHANGE_IDS=320,111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 10:52:12.478630 | crc | WORKSPACE=/home/zuul/workspace 2026-01-21 10:52:12.478637 | crc | XDG_SESSION_CLASS=user 2026-01-21 10:52:12.478643 | crc | SELINUX_ROLE_REQUESTED= 2026-01-21 10:52:12.478649 | crc | LESSOPEN=||/usr/bin/lesspipe.sh %s 2026-01-21 10:52:12.478654 | crc | USER=core 2026-01-21 10:52:12.478660 | crc | ZUUL_VOTING=True 2026-01-21 10:52:12.478665 | crc | BUILD_TIMEOUT=7200000 2026-01-21 10:52:12.478671 | crc | SELINUX_USE_CURRENT_RANGE= 2026-01-21 10:52:12.478676 | crc | SHLVL=1 2026-01-21 10:52:12.478683 | crc | ZUUL_PATCHSET=111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 10:52:12.478690 | crc | XDG_SESSION_ID=2 2026-01-21 10:52:12.478697 | crc | ZUUL_BRANCH=main 2026-01-21 10:52:12.478702 | crc | XDG_RUNTIME_DIR=/run/user/1000 2026-01-21 10:52:12.478709 | crc | SSH_CLIENT=38.102.83.114 38574 22 2026-01-21 10:52:12.478716 | crc | which_declare=declare -f 2026-01-21 10:52:12.478722 | crc | PATH=/var/home/core/.local/bin:/var/home/core/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin 2026-01-21 10:52:12.478728 | crc | SELINUX_LEVEL_REQUESTED= 2026-01-21 10:52:12.478735 | crc | ZUUL_CHANGE=320 2026-01-21 10:52:12.478742 | crc | DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus 2026-01-21 10:52:12.478749 | crc | ZUUL_UUID=4d9066f27a1f41f7af2512c243527c41 2026-01-21 10:52:12.478757 | crc | BASH_FUNC_which%%=() { ( alias; 2026-01-21 10:52:12.478766 | crc | eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot $@ 2026-01-21 10:52:12.478773 | crc | } 2026-01-21 10:52:12.445681 | controller | SHELL=/bin/bash 2026-01-21 10:52:12.445774 | controller | ZUUL_PROJECT=openstack-k8s-operators/watcher-operator 2026-01-21 10:52:12.445788 | controller | ZUUL_SHORT_PROJECT_NAME=watcher-operator 2026-01-21 10:52:12.445795 | controller | ZUUL_CHANGES=openstack-k8s-operators/watcher-operator:main:refs/changes/20/320/111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 10:52:12.445802 | controller | PWD=/home/zuul 2026-01-21 10:52:12.445808 | controller | ZUUL_PIPELINE=github-check 2026-01-21 10:52:12.445814 | controller | LOGNAME=zuul 2026-01-21 10:52:12.445820 | controller | XDG_SESSION_TYPE=tty 2026-01-21 10:52:12.445827 | controller | _=/usr/bin/env 2026-01-21 10:52:12.445833 | controller | MOTD_SHOWN=pam 2026-01-21 10:52:12.445839 | controller | HOME=/home/zuul 2026-01-21 10:52:12.445845 | controller | LANG=en_US.UTF-8 2026-01-21 10:52:12.445852 | controller | SSH_CONNECTION=38.102.83.114 49940 38.102.83.224 22 2026-01-21 10:52:12.445858 | controller | TOX_TESTENV_PASSENV=ZUUL_UUID ZUUL_PROJECT ZUUL_SHORT_PROJECT_NAME ZUUL_PIPELINE ZUUL_VOTING WORKSPACE BUILD_TIMEOUT ZUUL_BRANCH ZUUL_CHANGES ZUUL_CHANGE_IDS ZUUL_CHANGE ZUUL_PATCHSET 2026-01-21 10:52:12.445867 | controller | ZUUL_CHANGE_IDS=320,111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 10:52:12.445873 | controller | WORKSPACE=/home/zuul/workspace 2026-01-21 10:52:12.445880 | controller | XDG_SESSION_CLASS=user 2026-01-21 10:52:12.445886 | controller | SELINUX_ROLE_REQUESTED= 2026-01-21 10:52:12.445893 | controller | LESSOPEN=||/usr/bin/lesspipe.sh %s 2026-01-21 10:52:12.445900 | controller | USER=zuul 2026-01-21 10:52:12.445907 | controller | ZUUL_VOTING=True 2026-01-21 10:52:12.445913 | controller | BUILD_TIMEOUT=7200000 2026-01-21 10:52:12.445919 | controller | SELINUX_USE_CURRENT_RANGE= 2026-01-21 10:52:12.445925 | controller | SHLVL=1 2026-01-21 10:52:12.445932 | controller | ZUUL_PATCHSET=111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 10:52:12.445938 | controller | XDG_SESSION_ID=1 2026-01-21 10:52:12.445944 | controller | ZUUL_BRANCH=main 2026-01-21 10:52:12.445950 | controller | XDG_RUNTIME_DIR=/run/user/1000 2026-01-21 10:52:12.445957 | controller | SSH_CLIENT=38.102.83.114 49940 22 2026-01-21 10:52:12.445963 | controller | DEBUGINFOD_URLS=https://debuginfod.centos.org/ 2026-01-21 10:52:12.445970 | controller | DEBUGINFOD_IMA_CERT_PATH=/etc/keys/ima: 2026-01-21 10:52:12.445976 | controller | which_declare=declare -f 2026-01-21 10:52:12.445982 | controller | PATH=/home/zuul/.local/bin:/home/zuul/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin 2026-01-21 10:52:12.445989 | controller | SELINUX_LEVEL_REQUESTED= 2026-01-21 10:52:12.445995 | controller | ZUUL_CHANGE=320 2026-01-21 10:52:12.446001 | controller | DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus 2026-01-21 10:52:12.446007 | controller | ZUUL_UUID=4d9066f27a1f41f7af2512c243527c41 2026-01-21 10:52:12.446014 | controller | BASH_FUNC_which%%=() { ( alias; 2026-01-21 10:52:12.446020 | controller | eval ${which_declare} ) | /usr/bin/which --tty-only --read-alias --read-functions --show-tilde --show-dot $@ 2026-01-21 10:52:12.446026 | controller | } 2026-01-21 10:52:12.701485 | controller | ok: Runtime: 0:00:00.008568 2026-01-21 10:52:12.709106 | crc | ok: Runtime: 0:00:00.012100 2026-01-21 10:52:12.719267 | 2026-01-21 10:52:12.719387 | TASK [Workaround hardcoded /home/zuul/workspace in zuul_legacy_vars] 2026-01-21 10:52:12.771802 | controller | skipping: Conditional result was False 2026-01-21 10:52:12.803167 | crc | skipping: Conditional result was False 2026-01-21 10:52:12.809076 | 2026-01-21 10:52:12.809172 | TASK [Symlink /home/zuul-worker/workspace] 2026-01-21 10:52:13.408557 | controller | skipping: Conditional result was False 2026-01-21 10:52:13.408994 | crc | skipping: Conditional result was False 2026-01-21 10:52:13.415098 | 2026-01-21 10:52:13.415162 | TASK [Ensure legacy workspace directory] 2026-01-21 10:52:13.717730 | controller | changed 2026-01-21 10:52:13.724376 | crc | changed 2026-01-21 10:52:13.827242 | 2026-01-21 10:52:13.827337 | PLAY RECAP 2026-01-21 10:52:13.827378 | controller | ok: 14 changed: 11 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-01-21 10:52:13.827406 | crc | ok: 14 changed: 10 unreachable: 0 failed: 0 skipped: 5 rescued: 0 ignored: 0 2026-01-21 10:52:13.827425 | 2026-01-21 10:52:13.941024 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/ci-framework-rdo-base/pre.yaml@master] 2026-01-21 10:52:13.945352 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network.yml@master] 2026-01-21 10:52:15.761323 | 2026-01-21 10:52:15.761453 | PLAY [controller] 2026-01-21 10:52:15.797660 | 2026-01-21 10:52:15.797834 | TASK [Debug parameters] 2026-01-21 10:52:15.853001 | controller | ok 2026-01-21 10:52:15.867670 | 2026-01-21 10:52:15.867887 | TASK [Assert crc_ci_bootstrap_cloud_name is defined] 2026-01-21 10:52:15.911629 | controller | ok: All assertions passed 2026-01-21 10:52:15.916944 | 2026-01-21 10:52:15.917026 | TASK [Assert crc_ci_bootstrap_networking is defined and contains the minimum fields] 2026-01-21 10:52:15.958171 | controller | ok: All assertions passed 2026-01-21 10:52:15.966596 | 2026-01-21 10:52:15.966724 | TASK [Assert that the default network exists and has no VLAN tag] 2026-01-21 10:52:15.995498 | controller | ok: All assertions passed 2026-01-21 10:52:16.000790 | 2026-01-21 10:52:16.000860 | LOOP [Assert that all instances has a given fixed IP for each net and default net is present] 2026-01-21 10:52:16.101238 | controller | ok: All assertions passed 2026-01-21 10:52:16.115169 | controller | ok: All assertions passed 2026-01-21 10:52:16.121644 | 2026-01-21 10:52:16.121788 | LOOP [Assert that every host references an valid/declared network] 2026-01-21 10:52:16.202625 | controller | ok: All assertions passed 2026-01-21 10:52:16.202813 | controller | ok: All items complete 2026-01-21 10:52:16.202840 | 2026-01-21 10:52:16.207906 | controller | ok: All assertions passed 2026-01-21 10:52:16.221603 | 2026-01-21 10:52:16.221711 | LOOP [Assert that all instances has in range fixed IP for each net] 2026-01-21 10:52:16.310079 | controller | ok: All assertions passed 2026-01-21 10:52:16.339753 | controller | ok: All assertions passed 2026-01-21 10:52:16.358042 | 2026-01-21 10:52:16.358170 | TASK [Assert that all non-default nets has the vlan tag] 2026-01-21 10:52:16.421234 | controller | ok: All assertions passed 2026-01-21 10:52:16.432722 | 2026-01-21 10:52:16.432842 | TASK [Install required packages] 2026-01-21 10:52:47.466793 | controller | changed 2026-01-21 10:52:47.473051 | 2026-01-21 10:52:47.473144 | TASK [Create openstack config dir] 2026-01-21 10:52:47.932794 | controller | changed 2026-01-21 10:52:47.952821 | 2026-01-21 10:52:47.952954 | LOOP [Create ci dir in etc folder in the controller] 2026-01-21 10:52:48.327371 | controller -> 38.102.83.113 | changed: "crc" 2026-01-21 10:52:48.327866 | controller -> 38.102.83.113 | ok: All items complete 2026-01-21 10:52:48.327932 | 2026-01-21 10:52:48.609271 | controller -> 38.102.83.224 | changed: "controller" 2026-01-21 10:52:48.637811 | 2026-01-21 10:52:48.637940 | TASK [Set global resource names as facts] 2026-01-21 10:52:48.684350 | controller | ok 2026-01-21 10:52:48.689427 | 2026-01-21 10:52:48.689483 | TASK [Generate CA cert file] 2026-01-21 10:52:48.723196 | controller | skipping: Conditional result was False 2026-01-21 10:52:48.733974 | 2026-01-21 10:52:48.734056 | TASK [Update CA certs] 2026-01-21 10:52:49.271634 | controller | skipping: Conditional result was False 2026-01-21 10:52:49.280238 | 2026-01-21 10:52:49.280339 | TASK [Generate clouds config from cloud_secrets secret] 2026-01-21 10:52:50.333555 | controller | changed 2026-01-21 10:52:50.344552 | 2026-01-21 10:52:50.344640 | TASK [Create the default network] 2026-01-21 10:52:54.193396 | controller | admin_state_up: true 2026-01-21 10:52:54.193817 | controller | availability_zone_hints: 2026-01-21 10:52:54.194007 | controller | - nova 2026-01-21 10:52:54.194028 | controller | availability_zones: [] 2026-01-21 10:52:54.194327 | controller | created_at: '2026-01-21T10:52:53Z' 2026-01-21 10:52:54.194463 | controller | description: '' 2026-01-21 10:52:54.194586 | controller | dns_domain: '' 2026-01-21 10:52:54.194741 | controller | id: ccee4fed-07c7-4dcd-bc6b-76021b98eb53 2026-01-21 10:52:54.194936 | controller | ipv4_address_scope: null 2026-01-21 10:52:54.195114 | controller | ipv6_address_scope: null 2026-01-21 10:52:54.195307 | controller | is_default: false 2026-01-21 10:52:54.195481 | controller | is_vlan_qinq: null 2026-01-21 10:52:54.195631 | controller | is_vlan_transparent: true 2026-01-21 10:52:54.195772 | controller | l2_adjacency: true 2026-01-21 10:52:54.195986 | controller | mtu: 1500 2026-01-21 10:52:54.196157 | controller | name: zuul-ci-net-4d9066f2 2026-01-21 10:52:54.196336 | controller | port_security_enabled: false 2026-01-21 10:52:54.196528 | controller | project_id: 4b633c451ac74233be3721a3635275e5 2026-01-21 10:52:54.196701 | controller | provider:network_type: null 2026-01-21 10:52:54.196920 | controller | provider:physical_network: null 2026-01-21 10:52:54.197104 | controller | provider:segmentation_id: null 2026-01-21 10:52:54.197279 | controller | qos_policy_id: null 2026-01-21 10:52:54.197434 | controller | revision_number: 1 2026-01-21 10:52:54.197563 | controller | router:external: false 2026-01-21 10:52:54.197684 | controller | segments: null 2026-01-21 10:52:54.197826 | controller | shared: false 2026-01-21 10:52:54.197981 | controller | status: ACTIVE 2026-01-21 10:52:54.198160 | controller | subnets: [] 2026-01-21 10:52:54.198314 | controller | tags: [] 2026-01-21 10:52:54.198488 | controller | updated_at: '2026-01-21T10:52:53Z' 2026-01-21 10:52:54.393106 | controller | changed 2026-01-21 10:52:54.405102 | 2026-01-21 10:52:54.405274 | TASK [Create subnet - ipv4] 2026-01-21 10:52:57.217255 | controller | allocation_pools: 2026-01-21 10:52:57.217402 | controller | - end: 192.168.122.254 2026-01-21 10:52:57.217414 | controller | start: 192.168.122.2 2026-01-21 10:52:57.217426 | controller | cidr: 192.168.122.0/24 2026-01-21 10:52:57.217469 | controller | created_at: '2026-01-21T10:52:56Z' 2026-01-21 10:52:57.217491 | controller | description: '' 2026-01-21 10:52:57.217596 | controller | dns_nameservers: [] 2026-01-21 10:52:57.217795 | controller | dns_publish_fixed_ip: null 2026-01-21 10:52:57.217977 | controller | enable_dhcp: false 2026-01-21 10:52:57.218098 | controller | gateway_ip: 192.168.122.1 2026-01-21 10:52:57.218192 | controller | host_routes: [] 2026-01-21 10:52:57.218303 | controller | id: 29829789-a339-4200-85ef-2a8dc64a4ea5 2026-01-21 10:52:57.218423 | controller | ip_version: 4 2026-01-21 10:52:57.218527 | controller | ipv6_address_mode: null 2026-01-21 10:52:57.218620 | controller | ipv6_ra_mode: null 2026-01-21 10:52:57.218737 | controller | name: zuul-ci-subnet-4d9066f2 2026-01-21 10:52:57.218875 | controller | network_id: ccee4fed-07c7-4dcd-bc6b-76021b98eb53 2026-01-21 10:52:57.219022 | controller | project_id: 4b633c451ac74233be3721a3635275e5 2026-01-21 10:52:57.219126 | controller | revision_number: 0 2026-01-21 10:52:57.219218 | controller | segment_id: null 2026-01-21 10:52:57.219313 | controller | service_types: [] 2026-01-21 10:52:57.219408 | controller | subnetpool_id: null 2026-01-21 10:52:57.219494 | controller | tags: [] 2026-01-21 10:52:57.219602 | controller | updated_at: '2026-01-21T10:52:56Z' 2026-01-21 10:52:57.475898 | controller | changed 2026-01-21 10:52:57.488942 | 2026-01-21 10:52:57.489088 | TASK [Set yaml returned data for further usage] 2026-01-21 10:52:57.566694 | controller | ok 2026-01-21 10:52:57.574092 | 2026-01-21 10:52:57.574175 | TASK [Create router] 2026-01-21 10:52:58.112896 | controller | skipping: Conditional result was False 2026-01-21 10:52:58.125361 | 2026-01-21 10:52:58.125543 | TASK [Set router yaml returned data for further usage] 2026-01-21 10:52:58.172981 | controller | skipping: Conditional result was False 2026-01-21 10:52:58.185298 | 2026-01-21 10:52:58.185453 | TASK [Add router to subnet] 2026-01-21 10:52:58.771870 | controller | skipping: Conditional result was False 2026-01-21 10:52:58.785462 | 2026-01-21 10:52:58.785766 | LOOP [Create ports, VLANs and configure them on the instance] 2026-01-21 10:52:58.869796 | controller | ok: 2026-01-21 10:52:58.870050 | controller | { 2026-01-21 10:52:58.870085 | controller | "key": "controller", 2026-01-21 10:52:58.870110 | controller | "value": { 2026-01-21 10:52:58.870133 | controller | "networks": { 2026-01-21 10:52:58.870155 | controller | "default": { 2026-01-21 10:52:58.870177 | controller | "ip": "192.168.122.11" 2026-01-21 10:52:58.870199 | controller | } 2026-01-21 10:52:58.870221 | controller | } 2026-01-21 10:52:58.870242 | controller | } 2026-01-21 10:52:58.870263 | controller | } 2026-01-21 10:53:01.875001 | controller | ok: 2026-01-21 10:53:01.875161 | controller | { 2026-01-21 10:53:01.875212 | controller | "key": "crc", 2026-01-21 10:53:01.875252 | controller | "value": { 2026-01-21 10:53:01.875288 | controller | "networks": { 2026-01-21 10:53:01.875324 | controller | "default": { 2026-01-21 10:53:01.875360 | controller | "ip": "192.168.122.10" 2026-01-21 10:53:01.875395 | controller | }, 2026-01-21 10:53:01.875430 | controller | "internal-api": { 2026-01-21 10:53:01.875469 | controller | "ip": "172.17.0.5" 2026-01-21 10:53:01.875503 | controller | }, 2026-01-21 10:53:01.875538 | controller | "storage": { 2026-01-21 10:53:01.875572 | controller | "ip": "172.18.0.5" 2026-01-21 10:53:01.875607 | controller | }, 2026-01-21 10:53:01.875640 | controller | "tenant": { 2026-01-21 10:53:01.875674 | controller | "ip": "172.19.0.5" 2026-01-21 10:53:01.875753 | controller | } 2026-01-21 10:53:01.875790 | controller | } 2026-01-21 10:53:01.875824 | controller | } 2026-01-21 10:53:01.875859 | controller | } 2026-01-21 10:53:01.903854 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network-instance.yml 2026-01-21 10:53:01.910123 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network-instance.yml 2026-01-21 10:53:01.926120 | 2026-01-21 10:53:01.926243 | TASK [Create instance ports and attach if necessary] 2026-01-21 10:53:01.954567 | controller | ok 2026-01-21 10:53:02.019627 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network-instance-create-attach-port.yml 2026-01-21 10:53:02.025957 | 2026-01-21 10:53:02.026016 | TASK [Print the host networking data] 2026-01-21 10:53:02.065712 | controller | ok 2026-01-21 10:53:02.070629 | 2026-01-21 10:53:02.070719 | TASK [Create the private network parent port] 2026-01-21 10:53:04.910285 | controller | admin_state_up: true 2026-01-21 10:53:04.910606 | controller | allowed_address_pairs: [] 2026-01-21 10:53:04.910877 | controller | binding_host_id: null 2026-01-21 10:53:04.911043 | controller | binding_profile: {} 2026-01-21 10:53:04.911179 | controller | binding_vif_details: {} 2026-01-21 10:53:04.911310 | controller | binding_vif_type: null 2026-01-21 10:53:04.911474 | controller | binding_vnic_type: normal 2026-01-21 10:53:04.911621 | controller | created_at: '2026-01-21T10:53:04Z' 2026-01-21 10:53:04.911813 | controller | data_plane_status: null 2026-01-21 10:53:04.911987 | controller | description: '' 2026-01-21 10:53:04.912149 | controller | device_id: '' 2026-01-21 10:53:04.912337 | controller | device_owner: '' 2026-01-21 10:53:04.912515 | controller | device_profile: null 2026-01-21 10:53:04.912993 | controller | dns_assignment: 2026-01-21 10:53:04.913165 | controller | - fqdn: host-192-168-122-11.openstacklocal. 2026-01-21 10:53:04.913183 | controller | hostname: host-192-168-122-11 2026-01-21 10:53:04.913190 | controller | ip_address: 192.168.122.11 2026-01-21 10:53:04.913199 | controller | dns_domain: '' 2026-01-21 10:53:04.913320 | controller | dns_name: '' 2026-01-21 10:53:04.913445 | controller | extra_dhcp_opts: [] 2026-01-21 10:53:04.913847 | controller | fixed_ips: 2026-01-21 10:53:04.914009 | controller | - ip_address: 192.168.122.11 2026-01-21 10:53:04.914023 | controller | subnet_id: 29829789-a339-4200-85ef-2a8dc64a4ea5 2026-01-21 10:53:04.914033 | controller | hardware_offload_type: null 2026-01-21 10:53:04.914168 | controller | hints: '' 2026-01-21 10:53:04.914335 | controller | id: 6989b816-3d0d-4682-887c-aa01e2e291eb 2026-01-21 10:53:04.914473 | controller | ip_allocation: immediate 2026-01-21 10:53:04.914604 | controller | mac_address: fa:16:3e:be:97:29 2026-01-21 10:53:04.914857 | controller | name: controller-edcda38a-13f5-423b-8fec-d9fb09ea9acf 2026-01-21 10:53:04.915009 | controller | network_id: ccee4fed-07c7-4dcd-bc6b-76021b98eb53 2026-01-21 10:53:04.915155 | controller | numa_affinity_policy: null 2026-01-21 10:53:04.915390 | controller | port_security_enabled: false 2026-01-21 10:53:04.915587 | controller | project_id: 4b633c451ac74233be3721a3635275e5 2026-01-21 10:53:04.915830 | controller | propagate_uplink_status: null 2026-01-21 10:53:04.915996 | controller | resource_request: null 2026-01-21 10:53:04.916212 | controller | revision_number: 1 2026-01-21 10:53:04.916384 | controller | qos_network_policy_id: null 2026-01-21 10:53:04.916569 | controller | qos_policy_id: null 2026-01-21 10:53:04.916762 | controller | security_group_ids: [] 2026-01-21 10:53:04.916930 | controller | status: DOWN 2026-01-21 10:53:04.917086 | controller | tags: [] 2026-01-21 10:53:04.917268 | controller | trunk_details: null 2026-01-21 10:53:04.917429 | controller | trusted: null 2026-01-21 10:53:04.917616 | controller | updated_at: '2026-01-21T10:53:04Z' 2026-01-21 10:53:05.110760 | controller | changed 2026-01-21 10:53:05.123473 | 2026-01-21 10:53:05.123615 | TASK [Set common facts for further usage] 2026-01-21 10:53:05.223108 | controller | ok 2026-01-21 10:53:05.236565 | 2026-01-21 10:53:05.236741 | TASK [Create the trunk port] 2026-01-21 10:53:05.771986 | controller | skipping: Conditional result was False 2026-01-21 10:53:05.778351 | 2026-01-21 10:53:05.778423 | TASK [Avoid NetworkManager to automatically picked up the new interface] 2026-01-21 10:53:05.852882 | controller | skipping: Conditional result was False 2026-01-21 10:53:05.859454 | 2026-01-21 10:53:05.859521 | TASK [Refresh NetworkManager] 2026-01-21 10:53:05.924328 | controller | skipping: Conditional result was False 2026-01-21 10:53:05.930495 | 2026-01-21 10:53:05.930567 | TASK [Attach the port to the instance] 2026-01-21 10:53:12.508467 | controller | changed 2026-01-21 10:53:12.515913 | 2026-01-21 10:53:12.515997 | TASK [Wait till the interface exists] 2026-01-21 10:53:12.827935 | controller -> 38.102.83.224 | changed 2026-01-21 10:53:12.843768 | 2026-01-21 10:53:12.844149 | TASK [Prepare interface data to be used configuring trunk and vlan ifaces] 2026-01-21 10:53:12.978601 | controller | ok 2026-01-21 10:53:12.991363 | 2026-01-21 10:53:12.991513 | LOOP [Create VLANs ports and attach them to the trunk] 2026-01-21 10:53:13.036455 | 2026-01-21 10:53:13.036661 | LOOP [Prepare VLANs interface data to be used configuring] 2026-01-21 10:53:13.060373 | 2026-01-21 10:53:13.060540 | LOOP [Create NetworkManager configuration file for the trunk port] 2026-01-21 10:53:13.833538 | controller -> 38.102.83.224 | changed: 2026-01-21 10:53:13.833773 | controller -> 38.102.83.224 | { 2026-01-21 10:53:13.833801 | controller -> 38.102.83.224 | "key": "default", 2026-01-21 10:53:13.833818 | controller -> 38.102.83.224 | "value": { 2026-01-21 10:53:13.833835 | controller -> 38.102.83.224 | "connection": "ci-private-network", 2026-01-21 10:53:13.833852 | controller -> 38.102.83.224 | "gw": "192.168.122.1", 2026-01-21 10:53:13.833868 | controller -> 38.102.83.224 | "iface": "eth1", 2026-01-21 10:53:13.833883 | controller -> 38.102.83.224 | "ip": "192.168.122.11/24", 2026-01-21 10:53:13.833898 | controller -> 38.102.83.224 | "mac": "fa:16:3e:be:97:29", 2026-01-21 10:53:13.833913 | controller -> 38.102.83.224 | "mtu": "1500" 2026-01-21 10:53:13.833928 | controller -> 38.102.83.224 | } 2026-01-21 10:53:13.833943 | controller -> 38.102.83.224 | } 2026-01-21 10:53:13.833971 | 2026-01-21 10:53:13.859269 | 2026-01-21 10:53:13.859489 | TASK [Refresh NetworkManager] 2026-01-21 10:53:15.179803 | controller -> 38.102.83.224 | changed 2026-01-21 10:53:15.194569 | 2026-01-21 10:53:15.194706 | TASK [Debug fetch IP routes] 2026-01-21 10:53:15.525016 | controller -> 38.102.83.224 | changed 2026-01-21 10:53:15.533097 | 2026-01-21 10:53:15.533182 | TASK [Debug IP routes] 2026-01-21 10:53:15.578396 | controller -> 38.102.83.224 | ok 2026-01-21 10:53:15.585857 | 2026-01-21 10:53:15.585924 | TASK [Create instance ports and attach if necessary] 2026-01-21 10:53:15.609905 | controller | ok 2026-01-21 10:53:15.625489 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network-instance-create-attach-port.yml 2026-01-21 10:53:15.632187 | 2026-01-21 10:53:15.632261 | TASK [Print the host networking data] 2026-01-21 10:53:15.661522 | controller | ok 2026-01-21 10:53:15.668433 | 2026-01-21 10:53:15.668500 | TASK [Create the private network parent port] 2026-01-21 10:53:18.449421 | controller | admin_state_up: true 2026-01-21 10:53:18.449900 | controller | allowed_address_pairs: [] 2026-01-21 10:53:18.450364 | controller | binding_host_id: null 2026-01-21 10:53:18.450779 | controller | binding_profile: {} 2026-01-21 10:53:18.451093 | controller | binding_vif_details: {} 2026-01-21 10:53:18.451386 | controller | binding_vif_type: null 2026-01-21 10:53:18.451832 | controller | binding_vnic_type: normal 2026-01-21 10:53:18.452164 | controller | created_at: '2026-01-21T10:53:17Z' 2026-01-21 10:53:18.452458 | controller | data_plane_status: null 2026-01-21 10:53:18.452794 | controller | description: '' 2026-01-21 10:53:18.453105 | controller | device_id: '' 2026-01-21 10:53:18.453408 | controller | device_owner: '' 2026-01-21 10:53:18.453751 | controller | device_profile: null 2026-01-21 10:53:18.454597 | controller | dns_assignment: 2026-01-21 10:53:18.454948 | controller | - fqdn: host-192-168-122-10.openstacklocal. 2026-01-21 10:53:18.454992 | controller | hostname: host-192-168-122-10 2026-01-21 10:53:18.455018 | controller | ip_address: 192.168.122.10 2026-01-21 10:53:18.455039 | controller | dns_domain: '' 2026-01-21 10:53:18.455313 | controller | dns_name: '' 2026-01-21 10:53:18.455586 | controller | extra_dhcp_opts: [] 2026-01-21 10:53:18.456272 | controller | fixed_ips: 2026-01-21 10:53:18.456736 | controller | - ip_address: 192.168.122.10 2026-01-21 10:53:18.456779 | controller | subnet_id: 29829789-a339-4200-85ef-2a8dc64a4ea5 2026-01-21 10:53:18.456804 | controller | hardware_offload_type: null 2026-01-21 10:53:18.457113 | controller | hints: '' 2026-01-21 10:53:18.457510 | controller | id: 8e10dcc4-a879-4901-80f6-7b4acc1dd1f6 2026-01-21 10:53:18.457857 | controller | ip_allocation: immediate 2026-01-21 10:53:18.458158 | controller | mac_address: fa:16:3e:0e:f1:50 2026-01-21 10:53:18.458477 | controller | name: crc-57887b03-108e-4b07-83a9-2cba1ffe7256 2026-01-21 10:53:18.458834 | controller | network_id: ccee4fed-07c7-4dcd-bc6b-76021b98eb53 2026-01-21 10:53:18.459126 | controller | numa_affinity_policy: null 2026-01-21 10:53:18.459474 | controller | port_security_enabled: false 2026-01-21 10:53:18.459824 | controller | project_id: 4b633c451ac74233be3721a3635275e5 2026-01-21 10:53:18.460121 | controller | propagate_uplink_status: null 2026-01-21 10:53:18.460400 | controller | resource_request: null 2026-01-21 10:53:18.460797 | controller | revision_number: 1 2026-01-21 10:53:18.461092 | controller | qos_network_policy_id: null 2026-01-21 10:53:18.461359 | controller | qos_policy_id: null 2026-01-21 10:53:18.461630 | controller | security_group_ids: [] 2026-01-21 10:53:18.461915 | controller | status: DOWN 2026-01-21 10:53:18.462176 | controller | tags: [] 2026-01-21 10:53:18.462452 | controller | trunk_details: null 2026-01-21 10:53:18.462762 | controller | trusted: null 2026-01-21 10:53:18.463082 | controller | updated_at: '2026-01-21T10:53:17Z' 2026-01-21 10:53:18.707891 | controller | changed 2026-01-21 10:53:18.726598 | 2026-01-21 10:53:18.726905 | TASK [Set common facts for further usage] 2026-01-21 10:53:18.823659 | controller | ok 2026-01-21 10:53:18.838090 | 2026-01-21 10:53:18.838228 | TASK [Create the trunk port] 2026-01-21 10:53:19.390821 | controller | skipping: Conditional result was False 2026-01-21 10:53:19.404519 | 2026-01-21 10:53:19.404651 | TASK [Avoid NetworkManager to automatically picked up the new interface] 2026-01-21 10:53:19.492030 | controller | skipping: Conditional result was False 2026-01-21 10:53:19.506649 | 2026-01-21 10:53:19.506821 | TASK [Refresh NetworkManager] 2026-01-21 10:53:19.564981 | controller | skipping: Conditional result was False 2026-01-21 10:53:19.579149 | 2026-01-21 10:53:19.579294 | TASK [Attach the port to the instance] 2026-01-21 10:53:28.673652 | controller | changed 2026-01-21 10:53:28.679661 | 2026-01-21 10:53:28.679766 | TASK [Wait till the interface exists] 2026-01-21 10:53:29.092077 | controller -> 38.102.83.113 | changed 2026-01-21 10:53:29.100455 | 2026-01-21 10:53:29.100582 | TASK [Prepare interface data to be used configuring trunk and vlan ifaces] 2026-01-21 10:53:29.310651 | controller | ok 2026-01-21 10:53:29.320054 | 2026-01-21 10:53:29.320143 | LOOP [Create VLANs ports and attach them to the trunk] 2026-01-21 10:53:29.359649 | controller | skipping: Conditional result was False 2026-01-21 10:53:32.375873 | controller | skipping: Conditional result was False 2026-01-21 10:53:35.392182 | controller | skipping: Conditional result was False 2026-01-21 10:53:35.414219 | 2026-01-21 10:53:35.414430 | LOOP [Prepare VLANs interface data to be used configuring] 2026-01-21 10:53:35.559855 | controller | ok: 2026-01-21 10:53:35.560055 | controller | { 2026-01-21 10:53:35.560082 | controller | "key": "internal-api", 2026-01-21 10:53:35.560113 | controller | "value": { 2026-01-21 10:53:35.560130 | controller | "ip": "172.17.0.5" 2026-01-21 10:53:35.560146 | controller | } 2026-01-21 10:53:35.560161 | controller | } 2026-01-21 10:53:35.560186 | 2026-01-21 10:53:35.600824 | controller | ok: 2026-01-21 10:53:35.600919 | controller | { 2026-01-21 10:53:35.600943 | controller | "key": "storage", 2026-01-21 10:53:35.600961 | controller | "value": { 2026-01-21 10:53:35.600977 | controller | "ip": "172.18.0.5" 2026-01-21 10:53:35.600993 | controller | } 2026-01-21 10:53:35.601009 | controller | } 2026-01-21 10:53:35.637159 | controller | ok: 2026-01-21 10:53:35.637252 | controller | { 2026-01-21 10:53:35.637275 | controller | "key": "tenant", 2026-01-21 10:53:35.637292 | controller | "value": { 2026-01-21 10:53:35.637308 | controller | "ip": "172.19.0.5" 2026-01-21 10:53:35.637324 | controller | } 2026-01-21 10:53:35.637340 | controller | } 2026-01-21 10:53:35.661999 | 2026-01-21 10:53:35.662171 | LOOP [Create NetworkManager configuration file for the trunk port] 2026-01-21 10:53:36.649134 | controller -> 38.102.83.113 | changed: 2026-01-21 10:53:36.649401 | controller -> 38.102.83.113 | { 2026-01-21 10:53:36.649436 | controller -> 38.102.83.113 | "key": "default", 2026-01-21 10:53:36.649462 | controller -> 38.102.83.113 | "value": { 2026-01-21 10:53:36.649495 | controller -> 38.102.83.113 | "connection": "ci-private-network", 2026-01-21 10:53:36.649525 | controller -> 38.102.83.113 | "gw": "192.168.122.1", 2026-01-21 10:53:36.649552 | controller -> 38.102.83.113 | "iface": "ens7", 2026-01-21 10:53:36.649579 | controller -> 38.102.83.113 | "ip": "192.168.122.10/24", 2026-01-21 10:53:36.649606 | controller -> 38.102.83.113 | "mac": "fa:16:3e:0e:f1:50", 2026-01-21 10:53:36.649633 | controller -> 38.102.83.113 | "mtu": "1500" 2026-01-21 10:53:36.649660 | controller -> 38.102.83.113 | } 2026-01-21 10:53:36.649715 | controller -> 38.102.83.113 | } 2026-01-21 10:53:37.531771 | controller -> 38.102.83.113 | changed: 2026-01-21 10:53:37.531882 | controller -> 38.102.83.113 | { 2026-01-21 10:53:37.531904 | controller -> 38.102.83.113 | "key": "internal-api", 2026-01-21 10:53:37.531921 | controller -> 38.102.83.113 | "value": { 2026-01-21 10:53:37.531937 | controller -> 38.102.83.113 | "connection": "ci-private-network-20", 2026-01-21 10:53:37.531954 | controller -> 38.102.83.113 | "iface": "ens7.20", 2026-01-21 10:53:37.531969 | controller -> 38.102.83.113 | "ip": "172.17.0.5/24", 2026-01-21 10:53:37.531985 | controller -> 38.102.83.113 | "mac": "52:54:00:e6:d5:87", 2026-01-21 10:53:37.532000 | controller -> 38.102.83.113 | "mtu": "1496", 2026-01-21 10:53:37.532016 | controller -> 38.102.83.113 | "parent_iface": "ens7", 2026-01-21 10:53:37.532031 | controller -> 38.102.83.113 | "vlan": 20 2026-01-21 10:53:37.532046 | controller -> 38.102.83.113 | } 2026-01-21 10:53:37.532062 | controller -> 38.102.83.113 | } 2026-01-21 10:53:38.400157 | controller -> 38.102.83.113 | changed: 2026-01-21 10:53:38.400274 | controller -> 38.102.83.113 | { 2026-01-21 10:53:38.400298 | controller -> 38.102.83.113 | "key": "storage", 2026-01-21 10:53:38.400317 | controller -> 38.102.83.113 | "value": { 2026-01-21 10:53:38.400334 | controller -> 38.102.83.113 | "connection": "ci-private-network-21", 2026-01-21 10:53:38.400350 | controller -> 38.102.83.113 | "iface": "ens7.21", 2026-01-21 10:53:38.400367 | controller -> 38.102.83.113 | "ip": "172.18.0.5/24", 2026-01-21 10:53:38.400382 | controller -> 38.102.83.113 | "mac": "52:54:00:18:b8:51", 2026-01-21 10:53:38.400399 | controller -> 38.102.83.113 | "mtu": "1496", 2026-01-21 10:53:38.400414 | controller -> 38.102.83.113 | "parent_iface": "ens7", 2026-01-21 10:53:38.400429 | controller -> 38.102.83.113 | "vlan": 21 2026-01-21 10:53:38.400446 | controller -> 38.102.83.113 | } 2026-01-21 10:53:38.400461 | controller -> 38.102.83.113 | } 2026-01-21 10:53:39.238231 | controller -> 38.102.83.113 | changed: 2026-01-21 10:53:39.238348 | controller -> 38.102.83.113 | { 2026-01-21 10:53:39.238370 | controller -> 38.102.83.113 | "key": "tenant", 2026-01-21 10:53:39.238386 | controller -> 38.102.83.113 | "value": { 2026-01-21 10:53:39.238403 | controller -> 38.102.83.113 | "connection": "ci-private-network-22", 2026-01-21 10:53:39.238418 | controller -> 38.102.83.113 | "iface": "ens7.22", 2026-01-21 10:53:39.238488 | controller -> 38.102.83.113 | "ip": "172.19.0.5/24", 2026-01-21 10:53:39.238509 | controller -> 38.102.83.113 | "mac": "52:54:00:a2:bf:8d", 2026-01-21 10:53:39.238524 | controller -> 38.102.83.113 | "mtu": "1496", 2026-01-21 10:53:39.238540 | controller -> 38.102.83.113 | "parent_iface": "ens7", 2026-01-21 10:53:39.238554 | controller -> 38.102.83.113 | "vlan": 22 2026-01-21 10:53:39.238569 | controller -> 38.102.83.113 | } 2026-01-21 10:53:39.238583 | controller -> 38.102.83.113 | } 2026-01-21 10:53:39.260918 | 2026-01-21 10:53:39.261110 | TASK [Refresh NetworkManager] 2026-01-21 10:53:40.229297 | controller -> 38.102.83.113 | changed 2026-01-21 10:53:40.241157 | 2026-01-21 10:53:40.241269 | TASK [Debug fetch IP routes] 2026-01-21 10:53:40.596112 | controller -> 38.102.83.113 | changed 2026-01-21 10:53:40.606405 | 2026-01-21 10:53:40.606493 | TASK [Debug IP routes] 2026-01-21 10:53:40.665625 | controller -> 38.102.83.113 | ok 2026-01-21 10:53:40.674962 | 2026-01-21 10:53:40.675034 | TASK [Display some data about network ports] 2026-01-21 10:53:42.960963 | controller | +--------------------------------------+-------------------------------------------------+-------------------+-------------------------------------------------------------------------------+--------+ 2026-01-21 10:53:43.082556 | controller | | ID | Name | MAC Address | Fixed IP Addresses | Status | 2026-01-21 10:53:43.082633 | controller | +--------------------------------------+-------------------------------------------------+-------------------+-------------------------------------------------------------------------------+--------+ 2026-01-21 10:53:43.082650 | controller | | 0fc06de2-fd12-402d-8360-025c1b01e034 | | fa:16:3e:a8:a9:2a | | DOWN | 2026-01-21 10:53:43.082664 | controller | | 6989b816-3d0d-4682-887c-aa01e2e291eb | controller-edcda38a-13f5-423b-8fec-d9fb09ea9acf | fa:16:3e:be:97:29 | ip_address='192.168.122.11', subnet_id='29829789-a339-4200-85ef-2a8dc64a4ea5' | ACTIVE | 2026-01-21 10:53:43.082678 | controller | | 8e10dcc4-a879-4901-80f6-7b4acc1dd1f6 | crc-57887b03-108e-4b07-83a9-2cba1ffe7256 | fa:16:3e:0e:f1:50 | ip_address='192.168.122.10', subnet_id='29829789-a339-4200-85ef-2a8dc64a4ea5' | ACTIVE | 2026-01-21 10:53:43.082689 | controller | +--------------------------------------+-------------------------------------------------+-------------------+-------------------------------------------------------------------------------+--------+ 2026-01-21 10:53:43.214536 | controller | changed 2026-01-21 10:53:43.228120 | 2026-01-21 10:53:43.228263 | LOOP [Display server configuration] 2026-01-21 10:53:45.499487 | controller | changed 2026-01-21 10:53:45.500107 | controller | changed: All items complete 2026-01-21 10:53:45.500181 | 2026-01-21 10:53:51.370142 | controller | changed 2026-01-21 10:53:51.400708 | 2026-01-21 10:53:51.400847 | TASK [Fetch underneath provider DNSs] 2026-01-21 10:53:51.444660 | controller | ok 2026-01-21 10:53:51.456879 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network-fetch-provider-dns.yml 2026-01-21 10:53:51.473322 | 2026-01-21 10:53:51.473432 | TASK [Fetch the DNS list of the default interface] 2026-01-21 10:53:51.851470 | controller | ok 2026-01-21 10:53:51.866335 | 2026-01-21 10:53:51.866499 | TASK [Set provider DNS for further usage] 2026-01-21 10:53:51.928837 | controller | ok 2026-01-21 10:53:51.940596 | 2026-01-21 10:53:51.940783 | LOOP [Save networking data to file for further usage] 2026-01-21 10:53:52.876260 | controller -> 38.102.83.113 | changed: "crc" 2026-01-21 10:53:53.678076 | controller -> 38.102.83.224 | changed: "controller" 2026-01-21 10:53:53.701402 | 2026-01-21 10:53:53.701577 | TASK [Remove cloud_secrets file] 2026-01-21 10:53:54.010095 | controller | changed 2026-01-21 10:53:54.094805 | 2026-01-21 10:53:54.095056 | PLAY RECAP 2026-01-21 10:53:54.095107 | controller | ok: 48 changed: 22 unreachable: 0 failed: 0 skipped: 14 rescued: 0 ignored: 0 2026-01-21 10:53:54.095130 | 2026-01-21 10:53:54.232343 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/bootstrap-ci-network.yml@master] 2026-01-21 10:53:54.236982 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/prepare-crc.yaml@master] 2026-01-21 10:53:54.854994 | 2026-01-21 10:53:54.855109 | PLAY [Prefer ipv4 over ipv6] 2026-01-21 10:53:54.893076 | 2026-01-21 10:53:54.893192 | TASK [Configure /etc/gai.conf to prever ipv4 over ipv6] 2026-01-21 10:53:54.958995 | crc | skipping: Conditional result was False 2026-01-21 10:53:54.959564 | controller | skipping: Conditional result was False 2026-01-21 10:53:55.008265 | 2026-01-21 10:53:55.008375 | PLAY [Generic task for migration time to crc-cloud] 2026-01-21 10:53:55.027436 | 2026-01-21 10:53:55.027522 | TASK [Check if there is a image build date file] 2026-01-21 10:53:55.509743 | crc | ok 2026-01-21 10:53:55.520030 | 2026-01-21 10:53:55.520126 | TASK [Print image date] 2026-01-21 10:53:55.983247 | crc | Tue Jan 20 15:30:52 UTC 2026 | Image type: crc-cloud 2026-01-21 10:53:56.069910 | crc | ok 2026-01-21 10:53:56.077239 | 2026-01-21 10:53:56.077307 | TASK [Run crc-cloud preparation] 2026-01-21 10:53:56.124203 | crc | ok 2026-01-21 10:53:56.163216 | 2026-01-21 10:53:56.163314 | TASK [prepare-crc-cloud : Print build date] 2026-01-21 10:53:56.210439 | crc | ok 2026-01-21 10:53:56.226035 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/build_date.yaml 2026-01-21 10:53:56.243137 | 2026-01-21 10:53:56.243285 | TASK [prepare-crc-cloud : Check if there is a image build date file] 2026-01-21 10:53:56.543184 | crc | ok 2026-01-21 10:53:56.553714 | 2026-01-21 10:53:56.553817 | TASK [prepare-crc-cloud : Print image date if available] 2026-01-21 10:53:56.877756 | crc | Tue Jan 20 15:30:52 UTC 2026 | Image type: crc-cloud 2026-01-21 10:53:57.108316 | crc | ok 2026-01-21 10:53:57.124987 | 2026-01-21 10:53:57.125203 | TASK [prepare-crc-cloud : Tune Kubelet config file] 2026-01-21 10:53:57.162031 | crc | skipping: Conditional result was False 2026-01-21 10:53:57.172550 | 2026-01-21 10:53:57.172642 | TASK [prepare-crc-cloud : Reboot CRC host to get the network interfaces] 2026-01-21 10:53:57.207451 | crc | ok 2026-01-21 10:53:57.219256 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/reboot_crc.yaml 2026-01-21 10:53:57.234935 | 2026-01-21 10:53:57.235024 | TASK [prepare-crc-cloud : Reboot host after adding pull secret and changing kubelet node params] 2026-01-21 10:54:33.403455 | crc | changed 2026-01-21 10:54:33.418596 | 2026-01-21 10:54:33.418761 | TASK [Start Zuul console after reboot] 2026-01-21 10:54:33.468537 | crc | ok 2026-01-21 10:54:33.495299 | 2026-01-21 10:54:33.495543 | TASK [start-zuul-console : Start zuul_console daemon.] 2026-01-21 10:54:34.012193 | crc | ok 2026-01-21 10:54:34.039733 | 2026-01-21 10:54:34.039825 | TASK [prepare-crc-cloud : Prepare for executing crc-cloud] 2026-01-21 10:54:34.075652 | crc | ok 2026-01-21 10:54:34.088611 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/pre.yaml 2026-01-21 10:54:34.106624 | 2026-01-21 10:54:34.106752 | TASK [prepare-crc-cloud : Disable overwriting /etc/resolv.conf by the NetworkManager] 2026-01-21 10:54:35.186452 | crc | changed 2026-01-21 10:54:35.203585 | 2026-01-21 10:54:35.203779 | TASK [prepare-crc-cloud : Reload NetworkManager after creating custom rules] 2026-01-21 10:54:36.110979 | crc | changed 2026-01-21 10:54:36.130090 | 2026-01-21 10:54:36.130212 | TASK [prepare-crc-cloud : Inject pull secret into /var/lib/kubelet/config.json] 2026-01-21 10:54:37.057591 | crc | changed 2026-01-21 10:54:37.075191 | 2026-01-21 10:54:37.075392 | TASK [prepare-crc-cloud : Check if etcd slow profile set] 2026-01-21 10:54:37.101765 | crc | skipping: Conditional result was False 2026-01-21 10:54:37.123851 | 2026-01-21 10:54:37.124074 | TASK [prepare-crc-cloud : Use ramdisk for etcd] 2026-01-21 10:54:37.162019 | crc | ok 2026-01-21 10:54:37.177586 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/ramdisk.yaml 2026-01-21 10:54:37.194967 | 2026-01-21 10:54:37.195062 | TASK [prepare-crc-cloud : Move current etcd to other place] 2026-01-21 10:54:37.774324 | crc | ok 2026-01-21 10:54:37.785361 | 2026-01-21 10:54:37.785479 | TASK [prepare-crc-cloud : Mount ramdisk] 2026-01-21 10:54:38.254493 | crc | changed 2026-01-21 10:54:38.273103 | 2026-01-21 10:54:38.273302 | TASK [prepare-crc-cloud : Set proper permissions after mount] 2026-01-21 10:54:38.694154 | crc | changed 2026-01-21 10:54:38.710895 | 2026-01-21 10:54:38.711036 | TASK [prepare-crc-cloud : Copy content from old location to ramdisk] 2026-01-21 10:54:42.177033 | crc | changed 2026-01-21 10:54:42.184567 | 2026-01-21 10:54:42.184646 | TASK [prepare-crc-cloud : Set proper SELinux context] 2026-01-21 10:54:42.726979 | crc | ok 2026-01-21 10:54:42.743777 | 2026-01-21 10:54:42.743920 | TASK [prepare-crc-cloud : Block crc.testing domains] 2026-01-21 10:54:42.788908 | crc | ok 2026-01-21 10:54:42.809812 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/block_crc_domains.yaml 2026-01-21 10:54:42.832821 | 2026-01-21 10:54:42.832921 | LOOP [prepare-crc-cloud : Set crc.testing domains into /etc/hosts] 2026-01-21 10:54:43.316171 | crc | changed: line added 2026-01-21 10:54:43.606737 | crc | changed: line added 2026-01-21 10:54:43.885495 | crc | changed: line added 2026-01-21 10:54:44.145569 | crc | changed: line added 2026-01-21 10:54:44.448847 | crc | changed: line added 2026-01-21 10:54:44.731366 | crc | changed: line added 2026-01-21 10:54:44.999774 | crc | changed: line added 2026-01-21 10:54:45.274520 | crc | changed: line added 2026-01-21 10:54:45.297895 | 2026-01-21 10:54:45.297994 | TASK [prepare-crc-cloud : Get default interface name] 2026-01-21 10:54:45.618221 | crc | ens3 2026-01-21 10:54:45.870329 | crc | ok 2026-01-21 10:54:45.877841 | 2026-01-21 10:54:45.877916 | LOOP [prepare-crc-cloud : Block CRC domains also using iptables] 2026-01-21 10:54:46.596523 | crc | ok 2026-01-21 10:54:46.596953 | crc | changed: All items complete 2026-01-21 10:54:46.597014 | 2026-01-21 10:54:46.894054 | crc | ok 2026-01-21 10:54:47.192015 | crc | ok 2026-01-21 10:54:47.479231 | crc | ok 2026-01-21 10:54:47.772880 | crc | ok 2026-01-21 10:54:48.048617 | crc | ok 2026-01-21 10:54:48.351022 | crc | ok 2026-01-21 10:54:48.648299 | crc | ok 2026-01-21 10:54:48.678856 | 2026-01-21 10:54:48.679038 | TASK [prepare-crc-cloud : Set required facts] 2026-01-21 10:54:48.724010 | crc | ok 2026-01-21 10:54:48.741313 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/set_vars.yaml 2026-01-21 10:54:48.766103 | 2026-01-21 10:54:48.766267 | TASK [prepare-crc-cloud : Set required facts] 2026-01-21 10:54:48.859260 | crc | Output suppressed because no_log was given 2026-01-21 10:54:48.876072 | 2026-01-21 10:54:48.876188 | TASK [Start crc-cloud role when Zuul defined] 2026-01-21 10:54:48.922274 | crc | ok 2026-01-21 10:54:48.985942 | 2026-01-21 10:54:48.986054 | TASK [deploy-crc-cloud : Check if pull-secret is provided] 2026-01-21 10:54:49.010243 | crc | skipping: Conditional result was False 2026-01-21 10:54:49.018671 | 2026-01-21 10:54:49.018761 | TASK [deploy-crc-cloud : Create kubeconfig] 2026-01-21 10:54:49.053966 | crc | ok 2026-01-21 10:54:49.069804 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/kubeconfig.yaml 2026-01-21 10:54:49.088498 | 2026-01-21 10:54:49.088638 | TASK [deploy-crc-cloud : Create kube directory] 2026-01-21 10:54:49.346593 | crc | changed 2026-01-21 10:54:49.357548 | 2026-01-21 10:54:49.357657 | TASK [deploy-crc-cloud : Copy kubeconfig to user dir] 2026-01-21 10:54:49.826195 | crc | changed 2026-01-21 10:54:49.842209 | 2026-01-21 10:54:49.842345 | TASK [deploy-crc-cloud : Setup dnsmasq] 2026-01-21 10:54:49.883910 | crc | ok 2026-01-21 10:54:49.908602 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/dnsmasq.yaml 2026-01-21 10:54:49.928540 | 2026-01-21 10:54:49.928647 | TASK [deploy-crc-cloud : Disable overwriting /etc/resolv.conf by the NetworkManager] 2026-01-21 10:54:50.675996 | crc | ok 2026-01-21 10:54:50.694537 | 2026-01-21 10:54:50.694723 | TASK [deploy-crc-cloud : Restart NetworkManager when its needed] 2026-01-21 10:54:50.721969 | crc | skipping: Conditional result was False 2026-01-21 10:54:50.740442 | 2026-01-21 10:54:50.740595 | TASK [deploy-crc-cloud : Ensure /etc/resolv.conf is not a symlink and is writable] 2026-01-21 10:54:51.028069 | crc | ok 2026-01-21 10:54:51.046860 | 2026-01-21 10:54:51.047025 | TASK [deploy-crc-cloud : Create crc-dnsmasq.conf] 2026-01-21 10:54:52.030083 | crc | changed 2026-01-21 10:54:52.053405 | 2026-01-21 10:54:52.053592 | TASK [deploy-crc-cloud : Add domains into crc-dnsmasq.conf when alternative_domain] 2026-01-21 10:54:52.080477 | crc | skipping: Conditional result was False 2026-01-21 10:54:52.100133 | 2026-01-21 10:54:52.100367 | LOOP [deploy-crc-cloud : Set this host as first nameserver in /etc/resolv.conf] 2026-01-21 10:54:52.424642 | crc | changed: line replaced 2026-01-21 10:54:52.700584 | crc | ok 2026-01-21 10:54:52.984342 | crc | ok 2026-01-21 10:54:53.004719 | 2026-01-21 10:54:53.004940 | TASK [deploy-crc-cloud : Restart dnsmasq] 2026-01-21 10:54:54.933398 | crc | changed 2026-01-21 10:54:54.953543 | 2026-01-21 10:54:54.953727 | TASK [deploy-crc-cloud : Start kubelet] 2026-01-21 10:54:54.980923 | crc | ok 2026-01-21 10:54:54.995339 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/kubelet.yaml 2026-01-21 10:54:55.022086 | 2026-01-21 10:54:55.022206 | TASK [deploy-crc-cloud : Start and enable kubelet] 2026-01-21 10:55:09.479816 | crc | changed 2026-01-21 10:55:09.489602 | 2026-01-21 10:55:09.489699 | TASK [deploy-crc-cloud : Wait for port 6443 to be up] 2026-01-21 10:56:11.115977 | crc | ok 2026-01-21 10:56:11.135667 | 2026-01-21 10:56:11.135886 | TASK [deploy-crc-cloud : Wait for API to start before continue] 2026-01-21 10:56:11.879566 | crc | NAMESPACE NAME READY STATUS RESTARTS AGE 2026-01-21 10:56:11.881769 | crc | openshift-apiserver-operator openshift-apiserver-operator-796bbdcf4f-77kwx 0/1 Pending 0 25s 2026-01-21 10:56:11.881816 | crc | openshift-apiserver apiserver-76f77b778f-pxkk7 0/2 Pending 0 25s 2026-01-21 10:56:11.881824 | crc | openshift-authentication-operator authentication-operator-69f744f599-4kjnr 0/1 Pending 0 25s 2026-01-21 10:56:11.881832 | crc | openshift-authentication oauth-openshift-558db77b4-vwhv9 0/1 Pending 0 25s 2026-01-21 10:56:11.881840 | crc | openshift-cluster-machine-approver machine-approver-56656f9798-g66gm 0/2 Pending 0 25s 2026-01-21 10:56:11.881847 | crc | openshift-cluster-samples-operator cluster-samples-operator-665b6dd947-gqsng 0/2 Pending 0 25s 2026-01-21 10:56:11.881855 | crc | openshift-cluster-version cluster-version-operator-5c965bbfc6-t6bnk 0/1 Pending 0 25s 2026-01-21 10:56:11.881864 | crc | openshift-config-operator openshift-config-operator-7777fb866f-9fbhr 0/1 Pending 0 25s 2026-01-21 10:56:11.881871 | crc | openshift-console-operator console-operator-58897d9998-4d577 0/1 Pending 0 25s 2026-01-21 10:56:11.881879 | crc | openshift-console console-f9d7485db-7lrsj 0/1 Pending 0 25s 2026-01-21 10:56:11.881886 | crc | openshift-console downloads-7954f5f757-vw8cb 0/1 Pending 0 24s 2026-01-21 10:56:11.881928 | crc | openshift-controller-manager-operator openshift-controller-manager-operator-756b6f6bc6-jqrcn 0/1 Pending 0 24s 2026-01-21 10:56:11.881937 | crc | openshift-controller-manager controller-manager-879f6c89f-dz6wr 0/1 Pending 0 24s 2026-01-21 10:56:11.881945 | crc | openshift-dns-operator dns-operator-744455d44c-kk7wd 0/2 Pending 0 24s 2026-01-21 10:56:11.881953 | crc | openshift-dns node-resolver-rzmbp 0/1 Pending 0 25s 2026-01-21 10:56:11.881960 | crc | openshift-etcd-operator etcd-operator-b45778765-b7tzs 0/1 Pending 0 24s 2026-01-21 10:56:11.881968 | crc | openshift-etcd etcd-crc 0/5 Pending 0 28s 2026-01-21 10:56:11.881975 | crc | openshift-image-registry cluster-image-registry-operator-dc59b4c8b-cq2j2 0/1 Pending 0 24s 2026-01-21 10:56:11.881982 | crc | openshift-image-registry image-registry-697d97f7c8-m7dl4 0/1 Pending 0 24s 2026-01-21 10:56:11.881990 | crc | openshift-image-registry node-ca-jqsxs 0/1 Pending 0 24s 2026-01-21 10:56:11.881997 | crc | openshift-ingress-operator ingress-operator-5b745b69d9-4kbqk 0/2 Pending 0 24s 2026-01-21 10:56:11.882004 | crc | openshift-ingress router-default-5444994796-n2k47 0/1 Pending 0 24s 2026-01-21 10:56:11.882012 | crc | openshift-kube-apiserver-operator kube-apiserver-operator-766d6c64bb-5ml9s 0/1 Pending 0 24s 2026-01-21 10:56:11.882021 | crc | openshift-kube-apiserver kube-apiserver-crc 0/5 Pending 0 30s 2026-01-21 10:56:11.882050 | crc | openshift-kube-controller-manager-operator kube-controller-manager-operator-78b949d7b-28hkz 0/1 Pending 0 24s 2026-01-21 10:56:11.882082 | crc | openshift-kube-controller-manager kube-controller-manager-crc 0/4 Pending 0 30s 2026-01-21 10:56:11.882089 | crc | openshift-kube-scheduler-operator openshift-kube-scheduler-operator-5fdd9b5758-jb9kj 0/1 Pending 0 24s 2026-01-21 10:56:11.882097 | crc | openshift-kube-scheduler openshift-kube-scheduler-crc 0/3 Pending 0 0s 2026-01-21 10:56:11.882105 | crc | openshift-kube-storage-version-migrator-operator kube-storage-version-migrator-operator-b67b599dd-w2zsd 0/1 Pending 0 24s 2026-01-21 10:56:11.882112 | crc | openshift-kube-storage-version-migrator migrator-59844c95c7-hfk49 0/2 Pending 0 24s 2026-01-21 10:56:11.882119 | crc | openshift-machine-api control-plane-machine-set-operator-78cbb6b69f-2fd99 0/1 Pending 0 24s 2026-01-21 10:56:11.882133 | crc | openshift-machine-api machine-api-operator-5694c8668f-bmpxp 0/2 Pending 0 24s 2026-01-21 10:56:11.882140 | crc | openshift-machine-config-operator machine-config-controller-84d6567774-2d2sj 0/2 Pending 0 24s 2026-01-21 10:56:11.882147 | crc | openshift-machine-config-operator machine-config-daemon-rzs4q 0/2 Pending 0 24s 2026-01-21 10:56:11.882155 | crc | openshift-machine-config-operator machine-config-operator-74547568cd-q4cqt 0/2 Pending 0 23s 2026-01-21 10:56:11.882162 | crc | openshift-marketplace marketplace-operator-79b997595-8ht27 0/1 Pending 0 23s 2026-01-21 10:56:11.882169 | crc | openshift-multus multus-additional-cni-plugins-pbw2x 0/1 Pending 0 24s 2026-01-21 10:56:11.882177 | crc | openshift-multus multus-admission-controller-857f4d67dd-k9srb 0/2 Pending 0 23s 2026-01-21 10:56:11.882184 | crc | openshift-multus multus-hwzqb 0/1 Pending 0 24s 2026-01-21 10:56:11.882192 | crc | openshift-multus network-metrics-daemon-2txwq 0/2 Pending 0 24s 2026-01-21 10:56:11.882208 | crc | openshift-network-console networking-console-plugin-85b44fc459-gdk6g 1/1 Running 2 332d 2026-01-21 10:56:11.889129 | crc | openshift-network-diagnostics network-check-source-55646444c4-trplf 1/1 Running 2 332d 2026-01-21 10:56:11.889281 | crc | openshift-network-diagnostics network-check-target-xd92c 1/1 Running 2 332d 2026-01-21 10:56:11.889300 | crc | openshift-network-node-identity network-node-identity-vrzqb 2/2 Running 7 332d 2026-01-21 10:56:11.889314 | crc | openshift-network-operator iptables-alerter-4ln5h 1/1 Running 2 332d 2026-01-21 10:56:11.889331 | crc | openshift-network-operator network-operator-58b4c7f79c-55gtf 1/1 Running 4 332d 2026-01-21 10:56:11.889345 | crc | openshift-oauth-apiserver apiserver-7bbb656c7d-dnjzx 0/1 Pending 0 23s 2026-01-21 10:56:11.889533 | crc | openshift-operator-lifecycle-manager catalog-operator-68c6474976-krz9k 0/1 Pending 0 23s 2026-01-21 10:56:11.889618 | crc | openshift-operator-lifecycle-manager collect-profiles-29483205-hjb8l 0/1 Pending 0 24s 2026-01-21 10:56:11.889628 | crc | openshift-operator-lifecycle-manager olm-operator-6b444d44fb-z9nt4 0/1 Pending 0 23s 2026-01-21 10:56:11.889634 | crc | openshift-operator-lifecycle-manager package-server-manager-789f6589d5-fvwpn 0/2 Pending 0 23s 2026-01-21 10:56:11.889641 | crc | openshift-operator-lifecycle-manager packageserver-d55dfcdfc-5l4bq 0/1 Pending 0 23s 2026-01-21 10:56:11.889647 | crc | openshift-ovn-kubernetes ovnkube-control-plane-749d76644c-glmql 0/2 Pending 0 23s 2026-01-21 10:56:11.889653 | crc | openshift-ovn-kubernetes ovnkube-node-9hk9g 0/8 Pending 0 24s 2026-01-21 10:56:11.889661 | crc | openshift-route-controller-manager route-controller-manager-6576b87f9c-nrk92 0/1 Pending 0 23s 2026-01-21 10:56:11.889668 | crc | openshift-service-ca-operator service-ca-operator-777779d784-ntfc2 0/1 Pending 0 23s 2026-01-21 10:56:11.889676 | crc | openshift-service-ca service-ca-9c57cc56f-bhfrw 0/1 Pending 0 23s 2026-01-21 10:56:12.231787 | crc | ok 2026-01-21 10:56:12.251267 | 2026-01-21 10:56:12.251423 | TASK [deploy-crc-cloud : Replace default pubkey] 2026-01-21 10:56:12.295747 | crc | ok 2026-01-21 10:56:12.316897 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/pubkey.yaml 2026-01-21 10:56:12.357919 | 2026-01-21 10:56:12.358128 | TASK [deploy-crc-cloud : Check if id_rsa.pub exists] 2026-01-21 10:56:12.639337 | crc | ok 2026-01-21 10:56:12.649602 | 2026-01-21 10:56:12.649715 | TASK [deploy-crc-cloud : Read pub key] 2026-01-21 10:56:13.201627 | crc | skipping: Conditional result was False 2026-01-21 10:56:13.222007 | 2026-01-21 10:56:13.222177 | TASK [deploy-crc-cloud : Wait for machineconfig] 2026-01-21 10:56:13.249903 | crc | skipping: Conditional result was False 2026-01-21 10:56:13.269144 | 2026-01-21 10:56:13.269322 | TASK [deploy-crc-cloud : Patch machineconfig 99-master-ssh] 2026-01-21 10:56:13.828356 | crc | skipping: Conditional result was False 2026-01-21 10:56:13.848307 | 2026-01-21 10:56:13.848556 | TASK [deploy-crc-cloud : Accept certificate] 2026-01-21 10:56:13.896455 | crc | ok 2026-01-21 10:56:13.923067 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/accept_cert.yaml 2026-01-21 10:56:13.965346 | 2026-01-21 10:56:13.965468 | TASK [deploy-crc-cloud : Get csr in Pending state] 2026-01-21 10:56:14.519539 | crc | changed 2026-01-21 10:56:14.535625 | 2026-01-21 10:56:14.535866 | LOOP [deploy-crc-cloud : Accept OpenShift certificate if in Pending state] 2026-01-21 10:56:14.582819 | 2026-01-21 10:56:14.583411 | TASK [deploy-crc-cloud : Wait for cluster become healthy] 2026-01-21 10:56:14.628464 | crc | ok 2026-01-21 10:56:14.649055 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/wait_cluster_become_healthy.yaml 2026-01-21 10:56:14.698854 | 2026-01-21 10:56:14.699060 | TASK [deploy-crc-cloud : Check if specified OpenShift components are healthy etcd|openshift-apiserver] 2026-01-21 10:56:15.358333 | crc | True 2026-01-21 10:56:15.359157 | crc | False 2026-01-21 10:59:16.706061 | crc | changed 2026-01-21 10:59:16.723341 | 2026-01-21 10:59:16.723501 | TASK [deploy-crc-cloud : Output success message if components are healthy] 2026-01-21 10:59:16.777745 | crc | ok: OpenShift cluster components have become healthy in approximately 20 * 15 seconds. 2026-01-21 10:59:16.793873 | 2026-01-21 10:59:16.794016 | TASK [deploy-crc-cloud : Fail if any components are still unhealthy] 2026-01-21 10:59:16.821239 | crc | skipping: Conditional result was False 2026-01-21 10:59:16.841245 | 2026-01-21 10:59:16.841411 | TASK [deploy-crc-cloud : Set credentials] 2026-01-21 10:59:16.875448 | crc | ok 2026-01-21 10:59:16.894810 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/set_credentials.yaml 2026-01-21 10:59:16.928612 | 2026-01-21 10:59:16.928750 | TASK [deploy-crc-cloud : Ensure htpasswd.txt does not exists] 2026-01-21 10:59:18.050448 | crc | ok 2026-01-21 10:59:18.066374 | 2026-01-21 10:59:18.066506 | TASK [deploy-crc-cloud : Create temporary directory] 2026-01-21 10:59:19.363301 | crc | changed 2026-01-21 10:59:19.379032 | 2026-01-21 10:59:19.379183 | LOOP [deploy-crc-cloud : Get htpasswd] 2026-01-21 10:59:19.447648 | crc | Output suppressed because no_log was given 2026-01-21 10:59:19.459801 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/get_htpasswd.yaml 2026-01-21 10:59:19.461030 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/get_htpasswd.yaml 2026-01-21 10:59:19.462359 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/get_htpasswd.yaml 2026-01-21 10:59:19.477698 | 2026-01-21 10:59:19.477791 | TASK [deploy-crc-cloud : Get htpasswd for developer] 2026-01-21 10:59:54.434803 | crc | time="2026-01-21T10:59:54Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2026-01-21 10:59:54.442097 | crc | Trying to pull ghcr.io/crc-org/crc-cloud/httpd-tools:latest... 2026-01-21 10:59:55.077166 | crc | Getting image source signatures 2026-01-21 10:59:55.079040 | crc | Copying blob sha256:070cca9678d0cc5919ba8c1d8a2d3cb0e0481d641bf9300af526338bb91c9ac4 2026-01-21 10:59:55.080208 | crc | Copying blob sha256:bc46fc4c3e746a82292658b9b2cb6dc13c634a96e483d41d57ce74a5aae7c030 2026-01-21 11:00:05.315029 | crc | Copying config sha256:ddb130de62f40b931cccbd7055ad3e6881817ccc93fc70333d410dac87f51f1d 2026-01-21 11:00:05.326681 | crc | Writing manifest to image destination 2026-01-21 11:00:10.522298 | crc | changed 2026-01-21 11:00:10.540569 | 2026-01-21 11:00:10.540769 | TASK [deploy-crc-cloud : Get htpasswd for kubeadmin] 2026-01-21 11:00:11.012224 | crc | time="2026-01-21T11:00:11Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2026-01-21 11:00:16.463943 | crc | changed 2026-01-21 11:00:16.483560 | 2026-01-21 11:00:16.483791 | TASK [deploy-crc-cloud : Get htpasswd for redhat] 2026-01-21 11:00:16.974275 | crc | time="2026-01-21T11:00:16Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" 2026-01-21 11:00:21.689446 | crc | changed 2026-01-21 11:00:21.708160 | 2026-01-21 11:00:21.708314 | TASK [deploy-crc-cloud : Cleanup htpasswd.txt file] 2026-01-21 11:00:22.289050 | crc | changed 2026-01-21 11:00:22.301004 | 2026-01-21 11:00:22.301125 | TASK [deploy-crc-cloud : Create secret with generic htpass-secret] 2026-01-21 11:00:24.911959 | crc | ok 2026-01-21 11:00:24.931431 | 2026-01-21 11:00:24.931604 | TASK [deploy-crc-cloud : Replace htpass-secret] 2026-01-21 11:00:25.794363 | crc | secret/htpass-secret replaced 2026-01-21 11:00:25.880145 | crc | ok 2026-01-21 11:00:25.898076 | 2026-01-21 11:00:25.898221 | TASK [deploy-crc-cloud : Remove temporary directory] 2026-01-21 11:00:26.344401 | crc | changed 2026-01-21 11:00:26.351995 | 2026-01-21 11:00:26.352062 | TASK [deploy-crc-cloud : Replace default CA] 2026-01-21 11:00:26.384689 | crc | ok 2026-01-21 11:00:26.396010 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/replace_default_ca.yaml 2026-01-21 11:00:26.423655 | 2026-01-21 11:00:26.423743 | TASK [deploy-crc-cloud : Generate key] 2026-01-21 11:00:29.471409 | crc | changed 2026-01-21 11:00:29.488747 | 2026-01-21 11:00:29.489055 | TASK [deploy-crc-cloud : Generate cert] 2026-01-21 11:00:30.707394 | crc | changed 2026-01-21 11:00:30.723638 | 2026-01-21 11:00:30.723827 | TASK [deploy-crc-cloud : Generate csr] 2026-01-21 11:00:31.366117 | crc | .......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.....+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..........+......+...+..+.......+..+...+...............+......+.......+..+......+......+.....................+....+.....+...+..................+........................+.+..+......+.........................+..+.........+.+...+..+....+........+...............+.+............+........+.........+...+...+....+..+...+.......+...+......+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2026-01-21 11:00:31.454285 | crc | .........+.......+......+.........+.....+......+....+...........+...+.......+............+...+.....+............+...+...+.+......+..+...+.......+...+...+.....+.+.........+.........+...+..+.......+...+...+...+............+..+...+.......+..+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.................+......+.+........+.+..+...+..........+.....+.......+..+.+..+.......+.....+.......+......+...+...............+...........+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+....+...........+......+..........+.....+...+......+..........+.....................+........+...+.......+...+...+..+............+.............+........+.......+..............+.+........+.....................+.............+......+..+...+......+.+..+.......+.........+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2026-01-21 11:00:31.457807 | crc | ----- 2026-01-21 11:00:31.793169 | crc | changed 2026-01-21 11:00:31.854210 | 2026-01-21 11:00:31.854381 | TASK [deploy-crc-cloud : Generate user cert] 2026-01-21 11:00:32.279028 | crc | Certificate request self-signature ok 2026-01-21 11:00:32.289695 | crc | subject=O = ${GROUP}, CN = core 2026-01-21 11:00:32.403997 | crc | changed 2026-01-21 11:00:32.413187 | 2026-01-21 11:00:32.413312 | TASK [deploy-crc-cloud : Create configmap] 2026-01-21 11:00:33.531478 | crc | configmap/client-ca-custom created 2026-01-21 11:00:34.041493 | crc | ok 2026-01-21 11:00:34.060525 | 2026-01-21 11:00:34.060746 | TASK [deploy-crc-cloud : Patch apiserver] 2026-01-21 11:00:38.470706 | crc | apiserver.config.openshift.io/cluster patched 2026-01-21 11:00:39.045150 | crc | ok 2026-01-21 11:00:39.056125 | 2026-01-21 11:00:39.056225 | TASK [deploy-crc-cloud : Create configmap] 2026-01-21 11:00:41.216438 | crc | W0121 11:00:41.215787 13877 helpers.go:703] --dry-run is deprecated and can be replaced with --dry-run=client. 2026-01-21 11:00:41.976290 | crc | configmap/admin-kubeconfig-client-ca replaced 2026-01-21 11:00:42.466802 | crc | ok 2026-01-21 11:00:42.488123 | 2026-01-21 11:00:42.488305 | TASK [deploy-crc-cloud : Login to the OpenShift cluster] 2026-01-21 11:00:42.525038 | crc | ok 2026-01-21 11:00:42.544851 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/login.yaml 2026-01-21 11:00:42.591115 | 2026-01-21 11:00:42.591257 | TASK [deploy-crc-cloud : Try to login after all changes] 2026-01-21 11:00:44.460118 | crc | WARNING: Using insecure TLS client config. Setting this option is not supported! 2026-01-21 11:00:44.593162 | crc | 2026-01-21 11:00:44.593270 | crc | Unable to connect to the server: EOF 2026-01-21 11:01:49.666180 | crc | ok 2026-01-21 11:01:49.684169 | 2026-01-21 11:01:49.684350 | TASK [deploy-crc-cloud : Patch pull secret] 2026-01-21 11:01:49.728579 | crc | ok 2026-01-21 11:01:49.748312 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/patch_pull_secret.yaml 2026-01-21 11:01:49.795400 | 2026-01-21 11:01:49.795848 | TASK [deploy-crc-cloud : Patch pull secret] 2026-01-21 11:01:50.845707 | crc | Output suppressed because no_log was given 2026-01-21 11:01:50.862205 | 2026-01-21 11:01:50.862352 | TASK [deploy-crc-cloud : Wait for cluster become healthy after patching CA and pull secret] 2026-01-21 11:01:50.906003 | crc | ok 2026-01-21 11:01:50.926235 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/wait_cluster_become_healthy.yaml 2026-01-21 11:01:50.968160 | 2026-01-21 11:01:50.968270 | TASK [deploy-crc-cloud : Check if specified OpenShift components are healthy etcd|openshift-apiserver] 2026-01-21 11:01:51.440427 | crc | True 2026-01-21 11:01:51.441857 | crc | True 2026-01-21 11:01:51.508234 | crc | changed 2026-01-21 11:01:51.519495 | 2026-01-21 11:01:51.519725 | TASK [deploy-crc-cloud : Output success message if components are healthy] 2026-01-21 11:01:51.566257 | crc | ok: OpenShift cluster components have become healthy in approximately 20 * 15 seconds. 2026-01-21 11:01:51.576847 | 2026-01-21 11:01:51.576963 | TASK [deploy-crc-cloud : Fail if any components are still unhealthy] 2026-01-21 11:01:51.602740 | crc | skipping: Conditional result was False 2026-01-21 11:01:51.615859 | 2026-01-21 11:01:51.616005 | TASK [deploy-crc-cloud : Create certificate and patch secret] 2026-01-21 11:01:51.641470 | crc | skipping: Conditional result was False 2026-01-21 11:01:51.707020 | 2026-01-21 11:01:51.707152 | TASK [deploy-crc-cloud : Wait for cluster become healthy after adding domain] 2026-01-21 11:01:51.730300 | crc | ok 2026-01-21 11:01:51.739223 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/wait_cluster_become_healthy.yaml 2026-01-21 11:01:51.774040 | 2026-01-21 11:01:51.774145 | TASK [deploy-crc-cloud : Check if specified OpenShift components are healthy etcd|openshift-apiserver] 2026-01-21 11:01:52.288486 | crc | True 2026-01-21 11:01:52.289263 | crc | True 2026-01-21 11:01:52.332808 | crc | changed 2026-01-21 11:01:52.349631 | 2026-01-21 11:01:52.349827 | TASK [deploy-crc-cloud : Output success message if components are healthy] 2026-01-21 11:01:52.404021 | crc | ok: OpenShift cluster components have become healthy in approximately 20 * 15 seconds. 2026-01-21 11:01:52.419916 | 2026-01-21 11:01:52.420082 | TASK [deploy-crc-cloud : Fail if any components are still unhealthy] 2026-01-21 11:01:52.447068 | crc | skipping: Conditional result was False 2026-01-21 11:01:52.458923 | 2026-01-21 11:01:52.459021 | TASK [deploy-crc-cloud : Patch ingress config] 2026-01-21 11:01:52.484745 | crc | skipping: Conditional result was False 2026-01-21 11:01:52.505968 | 2026-01-21 11:01:52.506390 | TASK [deploy-crc-cloud : Patch api server] 2026-01-21 11:01:52.531964 | crc | skipping: Conditional result was False 2026-01-21 11:01:52.543730 | 2026-01-21 11:01:52.543837 | TASK [deploy-crc-cloud : Patch default route] 2026-01-21 11:01:52.569438 | crc | skipping: Conditional result was False 2026-01-21 11:01:52.581345 | 2026-01-21 11:01:52.581502 | TASK [deploy-crc-cloud : Wait for cluster become healthy after changing ingress api server and default route] 2026-01-21 11:01:52.613562 | crc | ok 2026-01-21 11:01:52.628665 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/src/github.com/crc-org/crc-cloud/ansible/roles/deploy-crc-cloud/tasks/wait_cluster_become_healthy.yaml 2026-01-21 11:01:52.670888 | 2026-01-21 11:01:52.671003 | TASK [deploy-crc-cloud : Check if specified OpenShift components are healthy authentication|console|etcd|ingress|openshift-apiserver] 2026-01-21 11:01:53.238230 | crc | False 2026-01-21 11:01:53.239144 | crc | True 2026-01-21 11:01:53.239163 | crc | True 2026-01-21 11:01:53.239169 | crc | True 2026-01-21 11:01:53.239175 | crc | True 2026-01-21 11:03:11.160593 | crc | changed 2026-01-21 11:03:11.175457 | 2026-01-21 11:03:11.175588 | TASK [deploy-crc-cloud : Output success message if components are healthy] 2026-01-21 11:03:11.240058 | crc | ok: OpenShift cluster components have become healthy in approximately 20 * 15 seconds. 2026-01-21 11:03:11.255283 | 2026-01-21 11:03:11.255438 | TASK [deploy-crc-cloud : Fail if any components are still unhealthy] 2026-01-21 11:03:11.292901 | crc | skipping: Conditional result was False 2026-01-21 11:03:11.312467 | 2026-01-21 11:03:11.312616 | TASK [deploy-crc-cloud : Get console route] 2026-01-21 11:03:11.339543 | crc | skipping: Conditional result was False 2026-01-21 11:03:11.372268 | 2026-01-21 11:03:11.372361 | TASK [Start crc-cloud role when Zuul is not defined] 2026-01-21 11:03:11.418205 | crc | skipping: Conditional result was False 2026-01-21 11:03:11.434532 | 2026-01-21 11:03:11.434811 | TASK [prepare-crc-cloud : Recreate kubelet config.json] 2026-01-21 11:03:11.479353 | crc | ok 2026-01-21 11:03:11.501290 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/inject-pull-secret.yaml 2026-01-21 11:03:11.536630 | 2026-01-21 11:03:11.536801 | TASK [prepare-crc-cloud : Fail when openshift_pull_sec not provided] 2026-01-21 11:03:11.602423 | crc | skipping: Conditional result was False 2026-01-21 11:03:11.619329 | 2026-01-21 11:03:11.619469 | TASK [prepare-crc-cloud : Cleanup file] 2026-01-21 11:03:12.182047 | crc | Output suppressed because no_log was given 2026-01-21 11:03:12.246259 | 2026-01-21 11:03:12.246392 | TASK [prepare-crc-cloud : Copy pull-secret to kubelet config.json] 2026-01-21 11:03:12.694163 | crc | Output suppressed because no_log was given 2026-01-21 11:03:12.709399 | 2026-01-21 11:03:12.709590 | TASK [prepare-crc-cloud : Replace singleqoute with doublequote] 2026-01-21 11:03:13.223312 | crc | changed: 42 replacements made 2026-01-21 11:03:13.238386 | 2026-01-21 11:03:13.238528 | TASK [prepare-crc-cloud : Remove spaces] 2026-01-21 11:03:13.712637 | crc | changed: 20 replacements made 2026-01-21 11:03:13.728852 | 2026-01-21 11:03:13.728992 | TASK [prepare-crc-cloud : Ensure you are logged in to the OpenShift] 2026-01-21 11:03:13.774267 | crc | ok 2026-01-21 11:03:13.789624 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/openshift_login.yaml 2026-01-21 11:03:13.821864 | 2026-01-21 11:03:13.821954 | TASK [prepare-crc-cloud : Login to the OpenShift] 2026-01-21 11:03:14.367385 | crc | WARNING: Using insecure TLS client config. Setting this option is not supported! 2026-01-21 11:03:14.555119 | crc | 2026-01-21 11:03:14.555220 | crc | Login successful. 2026-01-21 11:03:14.586763 | crc | 2026-01-21 11:03:14.586889 | crc | You have access to 65 projects, the list has been suppressed. You can list all projects with 'oc projects' 2026-01-21 11:03:14.609990 | crc | 2026-01-21 11:03:14.610320 | crc | Using project "default". 2026-01-21 11:03:14.865423 | crc | ok 2026-01-21 11:03:14.881454 | 2026-01-21 11:03:14.881671 | TASK [prepare-crc-cloud : Post tasks after crc-cloud script execution] 2026-01-21 11:03:14.924185 | crc | ok 2026-01-21 11:03:14.947353 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/post.yaml 2026-01-21 11:03:14.988602 | 2026-01-21 11:03:14.988777 | LOOP [prepare-crc-cloud : Set DNS nameservers provided by cloud provider] 2026-01-21 11:03:15.499033 | crc | changed: line added 2026-01-21 11:03:15.499744 | 2026-01-21 11:03:15.893884 | crc | changed: line added 2026-01-21 11:03:15.917169 | 2026-01-21 11:03:15.917418 | TASK [prepare-crc-cloud : Increase dnsmasq cache size] 2026-01-21 11:03:16.332861 | crc | changed: line added 2026-01-21 11:03:16.350651 | 2026-01-21 11:03:16.350830 | TASK [prepare-crc-cloud : Reload NetworkManager after adding cache and DNS server] 2026-01-21 11:03:17.495472 | crc | changed 2026-01-21 11:03:17.517583 | 2026-01-21 11:03:17.517817 | TASK [prepare-crc-cloud : Ensure kube config does not exists] 2026-01-21 11:03:17.866653 | crc | ok 2026-01-21 11:03:17.884582 | 2026-01-21 11:03:17.884794 | TASK [prepare-crc-cloud : Copy new generated KUBECONFIG] 2026-01-21 11:03:17.912202 | crc | skipping: Conditional result was False 2026-01-21 11:03:17.930972 | 2026-01-21 11:03:17.931180 | TASK [prepare-crc-cloud : Delete all openshift-marketplace pods] 2026-01-21 11:03:18.490492 | crc | pod "certified-operators-p5std" deleted 2026-01-21 11:03:18.500344 | crc | pod "community-operators-x5pnh" deleted 2026-01-21 11:03:18.511874 | crc | pod "marketplace-operator-79b997595-8ht27" deleted 2026-01-21 11:03:18.520780 | crc | pod "redhat-marketplace-xq95p" deleted 2026-01-21 11:03:18.532409 | crc | pod "redhat-operators-5kqxm" deleted 2026-01-21 11:03:20.546021 | crc | ok 2026-01-21 11:03:20.563611 | 2026-01-21 11:03:20.563818 | TASK [prepare-crc-cloud : Create workaround for expired cert] 2026-01-21 11:03:20.609310 | crc | ok 2026-01-21 11:03:20.631130 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/cluster_cert.yaml 2026-01-21 11:03:20.674045 | 2026-01-21 11:03:20.674223 | TASK [prepare-crc-cloud : Get OpenShift version] 2026-01-21 11:03:21.331056 | crc | 4.18.1 2026-01-21 11:03:21.720871 | crc | ok 2026-01-21 11:03:21.737760 | 2026-01-21 11:03:21.737916 | TASK [prepare-crc-cloud : Create service account for - auto-csr-approver] 2026-01-21 11:03:21.775311 | crc | skipping: Conditional result was False 2026-01-21 11:03:21.794797 | 2026-01-21 11:03:21.794962 | TASK [prepare-crc-cloud : Apply cluster cert approver] 2026-01-21 11:03:22.368087 | crc | skipping: Conditional result was False 2026-01-21 11:03:22.390645 | 2026-01-21 11:03:22.390895 | TASK [prepare-crc-cloud : Pause machineconfigpool] 2026-01-21 11:03:22.436608 | crc | ok 2026-01-21 11:03:22.458769 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/roles/prepare-crc-cloud/tasks/pause_mcp.yaml 2026-01-21 11:03:22.510318 | 2026-01-21 11:03:22.510568 | LOOP [prepare-crc-cloud : Pause the machineconfig configuration if mcp is not needed] 2026-01-21 11:03:23.146044 | crc | ok 2026-01-21 11:03:23.670946 | crc | ok 2026-01-21 11:03:23.703651 | 2026-01-21 11:03:23.703922 | TASK [prepare-crc-cloud : Print current machineconfigpool] 2026-01-21 11:03:24.352376 | crc | NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE 2026-01-21 11:03:24.352504 | crc | master rendered-master-f83a149c7490c3b0bf850df839837383 False True True 1 0 0 1 332d 2026-01-21 11:03:24.352518 | crc | worker rendered-worker-594cc07c9feefb5e37f2530ac457d36c True False False 0 0 0 0 332d 2026-01-21 11:03:24.776472 | crc | ok 2026-01-21 11:03:24.796625 | 2026-01-21 11:03:24.796904 | TASK [prepare-crc-cloud : Set low profile to etcd to avoid flakiness] 2026-01-21 11:03:24.828184 | crc | skipping: Conditional result was False 2026-01-21 11:03:24.846840 | 2026-01-21 11:03:24.847050 | TASK [prepare-crc-cloud : Fix machineconfigpool degradation] 2026-01-21 11:03:24.875199 | crc | skipping: Conditional result was False 2026-01-21 11:03:24.895342 | 2026-01-21 11:03:24.895760 | TASK [prepare-crc-cloud : Remove drain machineconfig for the node] 2026-01-21 11:03:24.923717 | crc | skipping: Conditional result was False 2026-01-21 11:03:24.936347 | 2026-01-21 11:03:24.936486 | TASK [prepare-crc-cloud : Post machineconfigpool tasks] 2026-01-21 11:03:24.971713 | crc | skipping: Conditional result was False 2026-01-21 11:03:24.984167 | 2026-01-21 11:03:24.984319 | TASK [prepare-crc-cloud : Enable monitoring and cluster-monitoring-operator] 2026-01-21 11:03:25.020186 | crc | skipping: Conditional result was False 2026-01-21 11:03:25.040381 | 2026-01-21 11:03:25.040518 | TASK [Run crc-extracted preparation] 2026-01-21 11:03:25.076879 | crc | skipping: Conditional result was False 2026-01-21 11:03:25.087934 | 2026-01-21 11:03:25.088096 | TASK [Fail when image build date does not exists] 2026-01-21 11:03:25.113771 | crc | skipping: Conditional result was False 2026-01-21 11:03:25.204607 | 2026-01-21 11:03:25.204737 | PLAY [Set IOPS and Read/Write disk limitation] 2026-01-21 11:03:25.234623 | 2026-01-21 11:03:25.234780 | TASK [Set limit for disk usage - IOPS and Read/Write] 2026-01-21 11:03:25.292525 | controller | ok 2026-01-21 11:03:25.323131 | crc | ok 2026-01-21 11:03:25.377532 | 2026-01-21 11:03:25.377672 | TASK [disk-usage-limitation : Get block device major:minor] 2026-01-21 11:03:25.864918 | controller | 252:0 2026-01-21 11:03:25.853755 | crc | 252:0 2026-01-21 11:03:25.955287 | crc | ok 2026-01-21 11:03:25.972966 | controller | ok 2026-01-21 11:03:25.990126 | 2026-01-21 11:03:25.990300 | LOOP [disk-usage-limitation : Ensure that some cgroup dirs exists] 2026-01-21 11:03:26.296871 | controller | ok: "/sys/fs/cgroup/init.scope" 2026-01-21 11:03:26.297593 | controller | ok: All items complete 2026-01-21 11:03:26.297718 | 2026-01-21 11:03:26.426370 | crc | ok: "/sys/fs/cgroup/init.scope" 2026-01-21 11:03:26.549578 | controller | changed: "/sys/fs/cgroup/machine.slice" 2026-01-21 11:03:26.840997 | controller | ok: "/sys/fs/cgroup/system.slice" 2026-01-21 11:03:26.856454 | crc | ok: "/sys/fs/cgroup/machine.slice" 2026-01-21 11:03:27.061076 | controller | ok: "/sys/fs/cgroup/user.slice" 2026-01-21 11:03:27.294085 | crc | ok: "/sys/fs/cgroup/system.slice" 2026-01-21 11:03:27.640509 | crc | ok: "/sys/fs/cgroup/user.slice" 2026-01-21 11:03:27.665206 | 2026-01-21 11:03:27.665385 | TASK [disk-usage-limitation : Ensure systemd configuration directory] 2026-01-21 11:03:28.005181 | controller | changed 2026-01-21 11:03:28.092967 | crc | ok 2026-01-21 11:03:28.108249 | 2026-01-21 11:03:28.108419 | TASK [disk-usage-limitation : Enable IO limit via override configuration] 2026-01-21 11:03:28.936981 | controller | changed 2026-01-21 11:03:29.433750 | crc | changed 2026-01-21 11:03:29.441945 | 2026-01-21 11:03:29.442045 | TASK [disk-usage-limitation : Flush handlers that will call systemd daemon-reload] 2026-01-21 11:03:29.453824 | 2026-01-21 11:03:29.453892 | TASK [disk-usage-limitation : Flush handlers that will call systemd daemon-reload] 2026-01-21 11:03:29.465476 | 2026-01-21 11:03:29.465550 | TASK [disk-usage-limitation : Reload systemd daemon after enabling IOAccounting] 2026-01-21 11:03:30.423314 | controller | ok 2026-01-21 11:03:31.765372 | crc | ok 2026-01-21 11:03:31.778462 | 2026-01-21 11:03:31.778630 | TASK [disk-usage-limitation : Wait for system.slice cgroup to have io.max file] 2026-01-21 11:03:32.142882 | crc | ok 2026-01-21 11:03:32.176571 | controller | ok 2026-01-21 11:03:32.190634 | 2026-01-21 11:03:32.190982 | LOOP [disk-usage-limitation : Set IOPS limit in the root cgroup] 2026-01-21 11:03:32.568867 | crc | ok 2026-01-21 11:03:32.569141 | crc | ok: All items complete 2026-01-21 11:03:32.569178 | 2026-01-21 11:03:32.577962 | controller | ok 2026-01-21 11:03:32.881183 | crc | ok 2026-01-21 11:03:33.189959 | controller | ok 2026-01-21 11:03:33.225456 | crc | ok 2026-01-21 11:03:33.534957 | crc | ok 2026-01-21 11:03:33.622990 | controller | ok 2026-01-21 11:03:33.871447 | controller | ok 2026-01-21 11:03:33.887988 | 2026-01-21 11:03:33.888158 | TASK [disk-usage-limitation : Verify the IOPS limit] 2026-01-21 11:03:34.225820 | controller | init 2026-01-21 11:03:34.229819 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.229875 | controller | machine 2026-01-21 11:03:34.229883 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.229889 | controller | system 2026-01-21 11:03:34.229896 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.229902 | controller | user 2026-01-21 11:03:34.229915 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.303149 | crc | init 2026-01-21 11:03:34.304917 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.305209 | crc | machine 2026-01-21 11:03:34.308152 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.308620 | crc | system 2026-01-21 11:03:34.310090 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.310184 | crc | user 2026-01-21 11:03:34.311676 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:34.483794 | controller | ok 2026-01-21 11:03:34.491259 | crc | ok 2026-01-21 11:03:34.498296 | 2026-01-21 11:03:34.498370 | TASK [disk-usage-limitation : Check if kubepods.slice exists] 2026-01-21 11:03:34.790203 | controller | ok 2026-01-21 11:03:34.876477 | crc | ok 2026-01-21 11:03:34.888775 | 2026-01-21 11:03:34.888931 | TASK [disk-usage-limitation : Set the limit] 2026-01-21 11:03:35.552587 | controller | skipping: Conditional result was False 2026-01-21 11:03:35.569040 | crc | ok 2026-01-21 11:03:35.582293 | 2026-01-21 11:03:35.582461 | TASK [disk-usage-limitation : Verify the IOPS limit - kubepods slice] 2026-01-21 11:03:36.004075 | crc | kubepods 2026-01-21 11:03:36.005433 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.184566 | controller | skipping: Conditional result was False 2026-01-21 11:03:36.201957 | crc | ok 2026-01-21 11:03:36.216451 | 2026-01-21 11:03:36.216632 | TASK [disk-usage-limitation : Print the current io.max value] 2026-01-21 11:03:36.300277 | controller | ok: 2026-01-21 11:03:36.300705 | controller | init 2026-01-21 11:03:36.300814 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.300875 | controller | machine 2026-01-21 11:03:36.300939 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.301073 | controller | system 2026-01-21 11:03:36.301119 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.301179 | controller | user 2026-01-21 11:03:36.301245 | controller | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.330203 | crc | ok: 2026-01-21 11:03:36.330349 | crc | init 2026-01-21 11:03:36.330408 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.330455 | crc | machine 2026-01-21 11:03:36.330498 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.330540 | crc | system 2026-01-21 11:03:36.330581 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.330622 | crc | user 2026-01-21 11:03:36.330663 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 kubepods 2026-01-21 11:03:36.330744 | crc | 252:0 rbps=262144000 wbps=262144000 riops=18000 wiops=18000 2026-01-21 11:03:36.405900 | 2026-01-21 11:03:36.405999 | PLAY RECAP 2026-01-21 11:03:36.406039 | controller | ok: 10 changed: 3 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-01-21 11:03:36.406064 | crc | ok: 108 changed: 37 unreachable: 0 failed: 0 skipped: 31 rescued: 0 ignored: 0 2026-01-21 11:03:36.406082 | 2026-01-21 11:03:36.541444 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/prepare-crc.yaml@master] 2026-01-21 11:03:36.547875 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/podman.yaml@master] 2026-01-21 11:03:37.153531 | 2026-01-21 11:03:37.153630 | PLAY [all,!compute] 2026-01-21 11:03:37.194579 | 2026-01-21 11:03:37.194740 | TASK [Enable container-tools rhel-modules for c8/crc node] 2026-01-21 11:03:37.765645 | controller | skipping: Conditional result was False 2026-01-21 11:03:37.766425 | crc | skipping: Conditional result was False 2026-01-21 11:03:37.775948 | 2026-01-21 11:03:37.776066 | TASK [Install podman] 2026-01-21 11:03:37.872511 | crc | skipping: Conditional result was False 2026-01-21 11:04:43.929059 | controller | changed 2026-01-21 11:04:43.942787 | 2026-01-21 11:04:43.942920 | TASK [Set var name for quay login zuul secret] 2026-01-21 11:04:44.015836 | controller | skipping: Conditional result was False 2026-01-21 11:04:44.037360 | crc | skipping: Conditional result was False 2026-01-21 11:04:44.048267 | 2026-01-21 11:04:44.048382 | TASK [Print the username] 2026-01-21 11:04:44.559125 | controller | openstack-k8s-operators+cirobot 2026-01-21 11:04:44.643873 | crc | skipping: Conditional result was False 2026-01-21 11:04:44.657908 | controller | ok: Runtime: 0:00:00.005140 2026-01-21 11:04:44.671985 | 2026-01-21 11:04:44.672185 | TASK [Fail when user or password is not set] 2026-01-21 11:04:44.752575 | controller | skipping: Conditional result was False 2026-01-21 11:04:44.774080 | crc | skipping: Conditional result was False 2026-01-21 11:04:44.790017 | 2026-01-21 11:04:44.790255 | TASK [Set vars for quay login] 2026-01-21 11:04:44.861453 | controller | Output suppressed because no_log was given 2026-01-21 11:04:44.879387 | 2026-01-21 11:04:44.879475 | LOOP [Perform container registry login(s) with podman] 2026-01-21 11:04:46.064467 | controller | Output suppressed because no_log was given 2026-01-21 11:04:46.078912 | 2026-01-21 11:04:46.079064 | LOOP [Perform container registry login(s) with buildah] 2026-01-21 11:04:46.152723 | controller | Output suppressed because no_log was given 2026-01-21 11:04:46.153074 | 2026-01-21 11:04:46.196398 | 2026-01-21 11:04:46.196619 | TASK [Set Insecure registry for content provider] 2026-01-21 11:04:46.290402 | crc | skipping: Conditional result was False 2026-01-21 11:04:46.766210 | controller | changed: Block inserted 2026-01-21 11:04:46.840879 | 2026-01-21 11:04:46.840987 | PLAY RECAP 2026-01-21 11:04:46.841036 | controller | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 4 rescued: 0 ignored: 0 2026-01-21 11:04:46.841070 | crc | ok: 0 changed: 0 unreachable: 0 failed: 0 skipped: 9 rescued: 0 ignored: 0 2026-01-21 11:04:46.841093 | 2026-01-21 11:04:46.954000 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/podman.yaml@master] 2026-01-21 11:04:46.958551 | PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/data_plane_adoption/dpa_RH_subscription_registry_login.yaml@master] 2026-01-21 11:04:47.629028 | 2026-01-21 11:04:47.629131 | PLAY [rh-subscription] 2026-01-21 11:04:47.629218 | 2026-01-21 11:04:47.629252 | PLAY RECAP 2026-01-21 11:04:47.629280 | 2026-01-21 11:04:47.704943 | PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/data_plane_adoption/dpa_RH_subscription_registry_login.yaml@master] 2026-01-21 11:04:47.714575 | PRE-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/multinode-customizations.yml@main] 2026-01-21 11:04:48.378178 | 2026-01-21 11:04:48.378290 | PLAY [Tweak CRC node] 2026-01-21 11:04:48.407817 | 2026-01-21 11:04:48.407929 | TASK [Load network parameters] 2026-01-21 11:04:49.064282 | crc | ok 2026-01-21 11:04:49.071612 | 2026-01-21 11:04:49.071709 | TASK [Check which dnsmasq config we must edit] 2026-01-21 11:04:49.517146 | crc | ok 2026-01-21 11:04:49.539500 | 2026-01-21 11:04:49.539735 | TASK [Ensure crc knows about its second NIC] 2026-01-21 11:04:50.465147 | crc | changed 2026-01-21 11:04:50.476029 | 2026-01-21 11:04:50.476163 | TASK [Ensure crc does not get "public" DNS] 2026-01-21 11:04:51.263916 | crc | changed 2026-01-21 11:04:51.282924 | 2026-01-21 11:04:51.283075 | TASK [Restart NetworkManager] 2026-01-21 11:04:52.582878 | crc | changed 2026-01-21 11:04:52.606549 | 2026-01-21 11:04:52.606786 | LOOP [Configure dns forwarders] 2026-01-21 11:04:53.181154 | crc | ok 2026-01-21 11:04:53.496241 | crc | ok 2026-01-21 11:04:53.516786 | 2026-01-21 11:04:53.516983 | TASK [Configure local DNS for CRC pod] 2026-01-21 11:04:54.085764 | crc | ok 2026-01-21 11:04:54.104304 | 2026-01-21 11:04:54.104542 | TASK [Configure dnsmasq listen-address to listen on both br-ex and ci-private-network] 2026-01-21 11:04:54.597989 | crc | changed: line added 2026-01-21 11:04:54.612832 | 2026-01-21 11:04:54.612965 | TASK [Restart native dnsmasq service] 2026-01-21 11:04:55.212140 | crc | changed 2026-01-21 11:04:55.231986 | 2026-01-21 11:04:55.232138 | TASK [Stop dnsmasq] 2026-01-21 11:04:55.261147 | crc | skipping: Conditional result was False 2026-01-21 11:04:55.279218 | 2026-01-21 11:04:55.279366 | TASK [Make sure that crc-dnsmasq is not running] 2026-01-21 11:04:55.307384 | crc | skipping: Conditional result was False 2026-01-21 11:04:55.324865 | 2026-01-21 11:04:55.325017 | TASK [Start dnsmasq] 2026-01-21 11:04:55.352804 | crc | skipping: Conditional result was False 2026-01-21 11:04:55.373218 | 2026-01-21 11:04:55.373385 | TASK [Wait for CRC to be ready] 2026-01-21 11:04:56.117102 | crc | WARNING: Using insecure TLS client config. Setting this option is not supported! 2026-01-21 11:04:56.302558 | crc | 2026-01-21 11:04:56.302654 | crc | Login successful. 2026-01-21 11:04:56.331377 | crc | 2026-01-21 11:04:56.331495 | crc | You have access to 65 projects, the list has been suppressed. You can list all projects with 'oc projects' 2026-01-21 11:04:56.350787 | crc | 2026-01-21 11:04:56.351032 | crc | Using project "default". 2026-01-21 11:04:56.428245 | crc | changed 2026-01-21 11:04:56.444907 | 2026-01-21 11:04:56.445059 | TASK [Set insecure registry on crc node] 2026-01-21 11:04:56.515815 | crc | ok 2026-01-21 11:04:56.540777 | crc | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/untrusted/project_0/github.com/openstack-k8s-operators/ci-framework/ci/playbooks/tasks/set_crc_insecure_registry.yml 2026-01-21 11:04:56.558537 | 2026-01-21 11:04:56.558658 | TASK [Patch the image.config.openshift.io resource to include insecure registry] 2026-01-21 11:04:57.122188 | crc | image.config.openshift.io/cluster patched 2026-01-21 11:04:57.608938 | crc | changed 2026-01-21 11:04:57.624005 | 2026-01-21 11:04:57.624143 | TASK [Patch the image.config.openshift.io resource to allow registries] 2026-01-21 11:04:58.224674 | crc | image.config.openshift.io/cluster patched 2026-01-21 11:04:58.719358 | crc | changed 2026-01-21 11:04:58.735413 | 2026-01-21 11:04:58.735594 | TASK [Ensure registries.conf.d exists] 2026-01-21 11:04:59.229654 | crc | ok 2026-01-21 11:04:59.245669 | 2026-01-21 11:04:59.245854 | TASK [Set Insecure registry for content provider] 2026-01-21 11:05:00.054640 | crc | changed: File created 2026-01-21 11:05:00.070196 | 2026-01-21 11:05:00.070331 | TASK [Restart crio] 2026-01-21 11:05:02.286432 | crc | changed 2026-01-21 11:05:02.343932 | 2026-01-21 11:05:02.344059 | PLAY [Tweak Controller] 2026-01-21 11:05:02.369227 | 2026-01-21 11:05:02.369364 | TASK [Install other packages] 2026-01-21 11:05:08.089381 | controller | changed 2026-01-21 11:05:08.103594 | 2026-01-21 11:05:08.103828 | LOOP [Ensure we know compute host keys] 2026-01-21 11:05:08.568714 | controller | changed 2026-01-21 11:05:08.569133 | controller | ok: All items complete 2026-01-21 11:05:08.569204 | 2026-01-21 11:05:08.928891 | controller | changed 2026-01-21 11:05:08.945345 | 2026-01-21 11:05:08.945565 | TASK [Generate an ssh keypair] 2026-01-21 11:05:09.278832 | controller | Generating public/private ecdsa key pair. 2026-01-21 11:05:09.278877 | controller | Your identification has been saved in /home/zuul/.ssh/id_cifw 2026-01-21 11:05:09.278888 | controller | Your public key has been saved in /home/zuul/.ssh/id_cifw.pub 2026-01-21 11:05:09.278896 | controller | The key fingerprint is: 2026-01-21 11:05:09.278904 | controller | SHA256:A2w23tPnceyAEKSsarI61fHaybXvy+OWP2rZJSDe6og zuul@np0005590255.novalocal 2026-01-21 11:05:09.278913 | controller | The key's randomart image is: 2026-01-21 11:05:09.278921 | controller | +---[ECDSA 256]---+ 2026-01-21 11:05:09.278928 | controller | | .o | 2026-01-21 11:05:09.278936 | controller | | o . . | 2026-01-21 11:05:09.278944 | controller | | O . | 2026-01-21 11:05:09.278952 | controller | | .= +.o.. . | 2026-01-21 11:05:09.278960 | controller | | ..o..Soo.+ o | 2026-01-21 11:05:09.278968 | controller | | ... . oo.o.=. | 2026-01-21 11:05:09.278976 | controller | |..o + o o +.o. | 2026-01-21 11:05:09.278983 | controller | |.+ ..+.o.* o | 2026-01-21 11:05:09.278991 | controller | |+. E ...*B=.. | 2026-01-21 11:05:09.279002 | controller | +----[SHA256]-----+ 2026-01-21 11:05:09.507853 | controller | changed 2026-01-21 11:05:09.521912 | 2026-01-21 11:05:09.522055 | TASK [Get public key] 2026-01-21 11:05:09.962861 | controller | ok 2026-01-21 11:05:09.971727 | 2026-01-21 11:05:09.971835 | TASK [Register pub key as a fact] 2026-01-21 11:05:10.035599 | controller | ok 2026-01-21 11:05:10.041744 | 2026-01-21 11:05:10.041810 | TASK [Load network parameters] 2026-01-21 11:05:10.418844 | controller | ok 2026-01-21 11:05:10.431127 | 2026-01-21 11:05:10.431263 | TASK [Check if cloud-init is overriding NM settings] 2026-01-21 11:05:10.812710 | controller | ok 2026-01-21 11:05:10.820375 | 2026-01-21 11:05:10.820457 | TASK [Remove cloud-init DNS override if present] 2026-01-21 11:05:10.845847 | controller | skipping: Conditional result was False 2026-01-21 11:05:10.854475 | 2026-01-21 11:05:10.854561 | TASK [Reload the NetworkManager to pick the changes] 2026-01-21 11:05:10.879420 | controller | skipping: Conditional result was False 2026-01-21 11:05:10.888196 | 2026-01-21 11:05:10.888292 | TASK [Get the default iface connection] 2026-01-21 11:05:11.234798 | controller | System eth0 2026-01-21 11:05:11.439157 | controller | changed 2026-01-21 11:05:11.446223 | 2026-01-21 11:05:11.446336 | TASK [Prepend CRC DNS server in the controllers default Network Manager connection configuation] 2026-01-21 11:05:12.274337 | controller | Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/5) 2026-01-21 11:05:12.495697 | controller | changed 2026-01-21 11:05:12.520016 | 2026-01-21 11:05:12.520130 | PLAY [Ensure persistent SSH access] 2026-01-21 11:05:12.553208 | 2026-01-21 11:05:12.553342 | TASK [Inject key in ansible_user] 2026-01-21 11:05:13.324915 | crc | changed 2026-01-21 11:05:13.479195 | controller | changed 2026-01-21 11:05:13.497263 | 2026-01-21 11:05:13.497424 | TASK [Inject key in root] 2026-01-21 11:05:14.056547 | crc | changed 2026-01-21 11:05:14.234045 | controller | changed 2026-01-21 11:05:14.328369 | 2026-01-21 11:05:14.328470 | PLAY [Add cloud-admin user on Compute] 2026-01-21 11:05:14.329435 | 2026-01-21 11:05:14.329483 | PLAY [Set some fancy hostname] 2026-01-21 11:05:14.346780 | 2026-01-21 11:05:14.346894 | TASK [Inject hostname in configuration file] 2026-01-21 11:05:15.559164 | controller | changed 2026-01-21 11:05:15.602998 | 2026-01-21 11:05:15.603115 | PLAY RECAP 2026-01-21 11:05:15.603174 | controller | ok: 12 changed: 8 unreachable: 0 failed: 0 skipped: 2 rescued: 0 ignored: 0 2026-01-21 11:05:15.603211 | crc | ok: 18 changed: 12 unreachable: 0 failed: 0 skipped: 3 rescued: 0 ignored: 0 2026-01-21 11:05:15.603238 | 2026-01-21 11:05:15.727565 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/multinode-customizations.yml@main] 2026-01-21 11:05:15.733165 | PRE-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/e2e-prepare.yml@main] 2026-01-21 11:05:16.434974 | 2026-01-21 11:05:16.435158 | PLAY [Run ci/playbooks/e2e-prepare.yml] 2026-01-21 11:05:16.468871 | 2026-01-21 11:05:16.468987 | TASK [Clone repos in the job workspace] 2026-01-21 11:05:16.490757 | controller | ok 2026-01-21 11:05:16.510951 | 2026-01-21 11:05:16.511215 | TASK [prepare-workspace : Start zuul_console daemon.] 2026-01-21 11:05:17.181796 | controller | ok 2026-01-21 11:05:17.188764 | 2026-01-21 11:05:17.188832 | TASK [prepare-workspace : Synchronize src repos to workspace directory.] 2026-01-21 11:05:31.049639 | controller | Output suppressed because no_log was given 2026-01-21 11:05:31.066089 | 2026-01-21 11:05:31.066176 | TASK [Create zuul-output directory] 2026-01-21 11:05:31.481614 | controller | ok 2026-01-21 11:05:31.498357 | 2026-01-21 11:05:31.498544 | TASK [Install required packages] 2026-01-21 11:05:32.785991 | controller | ok: Nothing to do 2026-01-21 11:05:32.802226 | 2026-01-21 11:05:32.802427 | TASK [Output pip related things] 2026-01-21 11:05:33.457290 | controller | pip 21.3.1 from /usr/lib/python3.9/site-packages/pip (python 3.9) 2026-01-21 11:05:33.857969 | controller | changed 2026-01-21 11:05:33.873625 | 2026-01-21 11:05:33.873855 | TASK [Install requirements] 2026-01-21 11:07:09.104550 | controller | changed 2026-01-21 11:07:09.134470 | 2026-01-21 11:07:09.134636 | TASK [Add cronjob to trigger job stats] 2026-01-21 11:07:09.183090 | controller | skipping: Conditional result was False 2026-01-21 11:07:09.204479 | 2026-01-21 11:07:09.204672 | LOOP [Construct project change list] 2026-01-21 11:07:09.268363 | controller | ok: 2026-01-21 11:07:09.268585 | controller | { 2026-01-21 11:07:09.268612 | controller | "branch": "main", 2026-01-21 11:07:09.268632 | controller | "change": "320", 2026-01-21 11:07:09.268652 | controller | "change_message": "Rabbitmq vhost and user support\n\nAdd new messagingBus and notificationsBus interfaces to hold cluster, user and vhost names for optional usage.\r\nThe controller adds these values to the TransportURL create request when present.\r\n\r\nAdditionally, we migrate RabbitMQ cluster name to RabbitMq config struct using DefaultRabbitMqConfig from infra-operator to automatically populate the new Cluster field from legacy RabbitMqClusterName.\r\n\r\nExample usage:\r\n\r\n```\r\n spec:\r\n messagingBus:\r\n cluster: rpc-rabbitmq\r\n user: rpc-user\r\n vhost: rpc-vhost\r\n notificationsBus:\r\n cluster: notifications-rabbitmq\r\n user: notifications-user\r\n vhost: notifications-vhost\r\n```\r\n\r\nJira: https://issues.redhat.com/browse/OSPRH-23882", 2026-01-21 11:07:09.268673 | controller | "change_url": "https://github.com/openstack-k8s-operators/watcher-operator/pull/320", 2026-01-21 11:07:09.268712 | controller | "commit_id": "111af6c57cd1be8032328ced6eb7a058a201c4b1", 2026-01-21 11:07:09.268730 | controller | "patchset": "111af6c57cd1be8032328ced6eb7a058a201c4b1", 2026-01-21 11:07:09.268746 | controller | "project": { 2026-01-21 11:07:09.268761 | controller | "canonical_hostname": "github.com", 2026-01-21 11:07:09.268777 | controller | "canonical_name": "github.com/openstack-k8s-operators/watcher-operator", 2026-01-21 11:07:09.268794 | controller | "name": "openstack-k8s-operators/watcher-operator", 2026-01-21 11:07:09.268809 | controller | "short_name": "watcher-operator", 2026-01-21 11:07:09.268824 | controller | "src_dir": "src/github.com/openstack-k8s-operators/watcher-operator" 2026-01-21 11:07:09.268840 | controller | }, 2026-01-21 11:07:09.268856 | controller | "topic": null 2026-01-21 11:07:09.268871 | controller | } 2026-01-21 11:07:09.334312 | 2026-01-21 11:07:09.334453 | PLAY [Run ci/playbooks/build_runner_image.yml] 2026-01-21 11:07:09.368495 | 2026-01-21 11:07:09.368603 | TASK [Filter out host if needed] 2026-01-21 11:07:09.379385 | controller | skipping: end_host conditional evaluated to False, continuing execution for controller 2026-01-21 11:07:09.385704 | 2026-01-21 11:07:09.385760 | TASK [Filter out host if needed] 2026-01-21 11:07:09.427311 | crc | skipping: end_host conditional evaluated to False, continuing execution for crc 2026-01-21 11:07:09.433088 | 2026-01-21 11:07:09.433170 | TASK [Get git tag for image tagging] 2026-01-21 11:07:10.003622 | controller | skipping: Conditional result was False 2026-01-21 11:07:10.004555 | crc | skipping: Conditional result was False 2026-01-21 11:07:10.017886 | 2026-01-21 11:07:10.018025 | TASK [Set openstack-runner image] 2026-01-21 11:07:10.061284 | controller | skipping: Conditional result was False 2026-01-21 11:07:10.073034 | crc | skipping: Conditional result was False 2026-01-21 11:07:10.086964 | 2026-01-21 11:07:10.087113 | TASK [Build openstack-runner image] 2026-01-21 11:07:10.124867 | controller | skipping: Conditional result was False 2026-01-21 11:07:10.125730 | crc | skipping: Conditional result was False 2026-01-21 11:07:10.134778 | 2026-01-21 11:07:10.134874 | TASK [Push openstack-runner image] 2026-01-21 11:07:10.158601 | controller | skipping: Conditional result was False 2026-01-21 11:07:10.170617 | crc | skipping: Conditional result was False 2026-01-21 11:07:10.188540 | 2026-01-21 11:07:10.188766 | TASK [Make sure ci-framework directory exists] 2026-01-21 11:07:10.216544 | controller | skipping: Conditional result was False 2026-01-21 11:07:10.227798 | crc | skipping: Conditional result was False 2026-01-21 11:07:10.237505 | 2026-01-21 11:07:10.237601 | TASK [Create EDPM ansible var file] 2026-01-21 11:07:10.261885 | controller | skipping: Conditional result was False 2026-01-21 11:07:10.283939 | crc | skipping: Conditional result was False 2026-01-21 11:07:10.370860 | 2026-01-21 11:07:10.370954 | PLAY RECAP 2026-01-21 11:07:10.370996 | controller | ok: 7 changed: 3 unreachable: 0 failed: 0 skipped: 7 rescued: 0 ignored: 0 2026-01-21 11:07:10.371022 | crc | ok: 0 changed: 0 unreachable: 0 failed: 0 skipped: 6 rescued: 0 ignored: 0 2026-01-21 11:07:10.371040 | 2026-01-21 11:07:10.496087 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/e2e-prepare.yml@main] 2026-01-21 11:07:10.502851 | PRE-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/dump_zuul_data.yml@main] 2026-01-21 11:07:11.105341 | 2026-01-21 11:07:11.105456 | PLAY [Run ci/playbooks/dump_zuul_data.yml] 2026-01-21 11:07:11.128985 | 2026-01-21 11:07:11.129084 | TASK [Create zuul-output directory] 2026-01-21 11:07:11.687195 | controller | changed 2026-01-21 11:07:11.701045 | 2026-01-21 11:07:11.701232 | TASK [Slurp Zuul inventory test] 2026-01-21 11:07:12.077613 | controller -> localhost | ok 2026-01-21 11:07:12.094179 | 2026-01-21 11:07:12.094330 | TASK [Save zuul inventory] 2026-01-21 11:07:13.223934 | controller | changed 2026-01-21 11:07:13.240542 | 2026-01-21 11:07:13.240746 | TASK [Save zuul vars without the change_message] 2026-01-21 11:07:14.126403 | controller | changed 2026-01-21 11:07:14.150461 | 2026-01-21 11:07:14.150557 | PLAY RECAP 2026-01-21 11:07:14.150604 | controller | ok: 4 changed: 3 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-01-21 11:07:14.150630 | 2026-01-21 11:07:14.239149 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/dump_zuul_data.yml@main] 2026-01-21 11:07:14.247237 | PRE-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/kuttl/kuttl-from-operator-deps.yaml@main] 2026-01-21 11:07:14.866094 | 2026-01-21 11:07:14.866210 | PLAY [Run ci/playbooks/kuttl/deploy-deps.yml] 2026-01-21 11:07:14.890438 | 2026-01-21 11:07:14.890539 | TASK [Run kuttl deploy-deps playbook] 2026-01-21 11:07:16.391331 | controller | 2026-01-21 11:07:17.782340 | controller | PLAY [Bootstrap playbook] ****************************************************** 2026-01-21 11:07:17.782418 | controller | 2026-01-21 11:07:17.782426 | controller | TASK [Gathering Facts ] ******************************************************** 2026-01-21 11:07:17.782433 | controller | Wednesday 21 January 2026 11:07:16 +0000 (0:00:00.062) 0:00:00.062 ***** 2026-01-21 11:07:17.782439 | controller | Wednesday 21 January 2026 11:07:16 +0000 (0:00:00.060) 0:00:00.060 ***** 2026-01-21 11:07:17.782457 | controller | ok: [controller] 2026-01-21 11:07:17.827601 | controller | 2026-01-21 11:07:17.827688 | controller | TASK [cifmw_setup : Set custom cifmw PATH reusable fact cifmw_path={{ ansible_user_dir }}/.crc/bin:{{ ansible_user_dir }}/.crc/bin/oc:{{ ansible_user_dir }}/bin:{{ ansible_env.PATH }}, cacheable=True] *** 2026-01-21 11:07:17.827704 | controller | Wednesday 21 January 2026 11:07:17 +0000 (0:00:01.391) 0:00:01.453 ***** 2026-01-21 11:07:17.827727 | controller | Wednesday 21 January 2026 11:07:17 +0000 (0:00:01.391) 0:00:01.452 ***** 2026-01-21 11:07:17.827750 | controller | ok: [controller] 2026-01-21 11:07:17.927188 | controller | 2026-01-21 11:07:17.927263 | controller | TASK [cifmw_setup : Get customized parameters ci_framework_params={{ 2026-01-21 11:07:17.927272 | controller | hostvars[inventory_hostname] | 2026-01-21 11:07:17.927287 | controller | dict2items | 2026-01-21 11:07:17.927294 | controller | selectattr("key", "match", 2026-01-21 11:07:17.927301 | controller | "^(cifmw|pre|post)_(?!install_yamls|openshift_token|openshift_login|openshift_kubeconfig).*") | 2026-01-21 11:07:17.927308 | controller | list | items2dict 2026-01-21 11:07:17.927314 | controller | }}] *** 2026-01-21 11:07:17.927321 | controller | Wednesday 21 January 2026 11:07:17 +0000 (0:00:00.044) 0:00:01.498 ***** 2026-01-21 11:07:17.927328 | controller | Wednesday 21 January 2026 11:07:17 +0000 (0:00:00.044) 0:00:01.496 ***** 2026-01-21 11:07:17.927344 | controller | ok: [controller] 2026-01-21 11:07:18.402475 | controller | 2026-01-21 11:07:18.402516 | controller | TASK [install_ca : Ensure target directory exists path={{ cifmw_install_ca_trust_dir }}, state=directory, mode=0755] *** 2026-01-21 11:07:18.402525 | controller | Wednesday 21 January 2026 11:07:17 +0000 (0:00:00.096) 0:00:01.594 ***** 2026-01-21 11:07:18.402532 | controller | Wednesday 21 January 2026 11:07:17 +0000 (0:00:00.096) 0:00:01.593 ***** 2026-01-21 11:07:18.402544 | controller | ok: [controller] 2026-01-21 11:07:18.432209 | controller | 2026-01-21 11:07:18.432248 | controller | TASK [install_ca : Install internal CA from url url={{ cifmw_install_ca_url }}, dest={{ cifmw_install_ca_trust_dir }}, validate_certs={{ cifmw_install_ca_url_validate_certs | default(omit) }}, mode=0644] *** 2026-01-21 11:07:18.432257 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.478) 0:00:02.073 ***** 2026-01-21 11:07:18.432264 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.478) 0:00:02.071 ***** 2026-01-21 11:07:18.432282 | controller | skipping: [controller] 2026-01-21 11:07:18.464909 | controller | 2026-01-21 11:07:18.464954 | controller | TASK [install_ca : Install custom CA bundle from inline dest={{ cifmw_install_ca_trust_dir }}/cifmw_inline_ca_bundle.crt, content={{ cifmw_install_ca_bundle_inline }}, mode=0644] *** 2026-01-21 11:07:18.464963 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.030) 0:00:02.103 ***** 2026-01-21 11:07:18.464971 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.030) 0:00:02.102 ***** 2026-01-21 11:07:18.464983 | controller | skipping: [controller] 2026-01-21 11:07:18.497840 | controller | 2026-01-21 11:07:18.497878 | controller | TASK [install_ca : Install custom CA bundle from file dest={{ cifmw_install_ca_trust_dir }}/{{ cifmw_install_ca_bundle_src | basename }}, src={{ cifmw_install_ca_bundle_src }}, mode=0644] *** 2026-01-21 11:07:18.497888 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.032) 0:00:02.136 ***** 2026-01-21 11:07:18.497896 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.032) 0:00:02.135 ***** 2026-01-21 11:07:18.497907 | controller | skipping: [controller] 2026-01-21 11:07:20.353523 | controller | 2026-01-21 11:07:20.353810 | controller | TASK [install_ca : Update ca bundle _raw_params=update-ca-trust] *************** 2026-01-21 11:07:20.353843 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.031) 0:00:02.168 ***** 2026-01-21 11:07:20.353865 | controller | Wednesday 21 January 2026 11:07:18 +0000 (0:00:00.032) 0:00:02.168 ***** 2026-01-21 11:07:20.353891 | controller | changed: [controller] 2026-01-21 11:07:21.125215 | controller | 2026-01-21 11:07:21.125270 | controller | TASK [repo_setup : Ensure directories are present path={{ cifmw_repo_setup_basedir }}/{{ item }}, state=directory, mode=0755] *** 2026-01-21 11:07:21.125280 | controller | Wednesday 21 January 2026 11:07:20 +0000 (0:00:01.855) 0:00:04.024 ***** 2026-01-21 11:07:21.125289 | controller | Wednesday 21 January 2026 11:07:20 +0000 (0:00:01.854) 0:00:04.022 ***** 2026-01-21 11:07:21.125309 | controller | changed: [controller] => (item=tmp) 2026-01-21 11:07:22.401622 | controller | changed: [controller] => (item=artifacts/repositories) 2026-01-21 11:07:22.401676 | controller | changed: [controller] => (item=venv/repo_setup) 2026-01-21 11:07:22.401685 | controller | 2026-01-21 11:07:22.401692 | controller | TASK [repo_setup : Make sure git-core package is installed name=git-core, state=present] *** 2026-01-21 11:07:22.401699 | controller | Wednesday 21 January 2026 11:07:21 +0000 (0:00:00.772) 0:00:04.796 ***** 2026-01-21 11:07:22.401706 | controller | Wednesday 21 January 2026 11:07:21 +0000 (0:00:00.772) 0:00:04.794 ***** 2026-01-21 11:07:22.401738 | controller | ok: [controller] 2026-01-21 11:07:24.263637 | controller | 2026-01-21 11:07:24.263788 | controller | TASK [repo_setup : Get repo-setup repository accept_hostkey=True, dest={{ cifmw_repo_setup_basedir }}/tmp/repo-setup, repo={{ cifmw_repo_setup_src }}] *** 2026-01-21 11:07:24.263807 | controller | Wednesday 21 January 2026 11:07:22 +0000 (0:00:01.276) 0:00:06.072 ***** 2026-01-21 11:07:24.263822 | controller | Wednesday 21 January 2026 11:07:22 +0000 (0:00:01.276) 0:00:06.071 ***** 2026-01-21 11:07:24.263847 | controller | changed: [controller] 2026-01-21 11:07:33.840709 | controller | 2026-01-21 11:07:33.840915 | controller | TASK [repo_setup : Initialize python venv and install requirements virtualenv={{ cifmw_repo_setup_venv }}, requirements={{ cifmw_repo_setup_basedir }}/tmp/repo-setup/requirements.txt, virtualenv_command=python3 -m venv --system-site-packages --upgrade-deps] *** 2026-01-21 11:07:33.840941 | controller | Wednesday 21 January 2026 11:07:24 +0000 (0:00:01.862) 0:00:07.934 ***** 2026-01-21 11:07:33.840956 | controller | Wednesday 21 January 2026 11:07:24 +0000 (0:00:01.862) 0:00:07.933 ***** 2026-01-21 11:07:33.840982 | controller | changed: [controller] 2026-01-21 11:07:34.774851 | controller | 2026-01-21 11:07:34.774901 | controller | TASK [repo_setup : Install repo-setup package chdir={{ cifmw_repo_setup_basedir }}/tmp/repo-setup, creates={{ cifmw_repo_setup_venv }}/bin/repo-setup, _raw_params={{ cifmw_repo_setup_venv }}/bin/python setup.py install] *** 2026-01-21 11:07:34.774928 | controller | Wednesday 21 January 2026 11:07:33 +0000 (0:00:09.576) 0:00:17.511 ***** 2026-01-21 11:07:34.774938 | controller | Wednesday 21 January 2026 11:07:33 +0000 (0:00:09.576) 0:00:17.510 ***** 2026-01-21 11:07:34.774961 | controller | changed: [controller] 2026-01-21 11:07:34.807912 | controller | 2026-01-21 11:07:34.807969 | controller | TASK [repo_setup : Set cifmw_repo_setup_dlrn_hash_tag from content provider cifmw_repo_setup_dlrn_hash_tag={{ content_provider_dlrn_md5_hash }}] *** 2026-01-21 11:07:34.807995 | controller | Wednesday 21 January 2026 11:07:34 +0000 (0:00:00.934) 0:00:18.446 ***** 2026-01-21 11:07:34.808004 | controller | Wednesday 21 January 2026 11:07:34 +0000 (0:00:00.934) 0:00:18.444 ***** 2026-01-21 11:07:34.808019 | controller | skipping: [controller] 2026-01-21 11:07:35.514090 | controller | 2026-01-21 11:07:35.514147 | controller | TASK [repo_setup : Run repo-setup _raw_params={{ cifmw_repo_setup_venv }}/bin/repo-setup {{ cifmw_repo_setup_promotion }} {{ cifmw_repo_setup_additional_repos }} -d {{ cifmw_repo_setup_os_release }}{{ cifmw_repo_setup_dist_major_version }} -b {{ cifmw_repo_setup_branch }} --rdo-mirror {{ cifmw_repo_setup_rdo_mirror }} {% if cifmw_repo_setup_dlrn_hash_tag | length > 0 %} --dlrn-hash-tag {{ cifmw_repo_setup_dlrn_hash_tag }} {% endif %} -o {{ cifmw_repo_setup_output }}] *** 2026-01-21 11:07:35.514165 | controller | Wednesday 21 January 2026 11:07:34 +0000 (0:00:00.032) 0:00:18.479 ***** 2026-01-21 11:07:35.514181 | controller | Wednesday 21 January 2026 11:07:34 +0000 (0:00:00.032) 0:00:18.477 ***** 2026-01-21 11:07:35.514206 | controller | changed: [controller] 2026-01-21 11:07:35.552194 | controller | 2026-01-21 11:07:35.552236 | controller | TASK [repo_setup : Get component repo url={{ cifmw_repo_setup_dlrn_uri }}/{{ cifmw_repo_setup_os_release }}{{ cifmw_repo_setup_dist_major_version }}-{{ cifmw_repo_setup_branch }}/component/{{ cifmw_repo_setup_component_name }}/{{ cifmw_repo_setup_component_promotion_tag }}/delorean.repo, dest={{ cifmw_repo_setup_output }}/{{ cifmw_repo_setup_component_name }}_{{ cifmw_repo_setup_component_promotion_tag }}_delorean.repo, mode=0644] *** 2026-01-21 11:07:35.552245 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.706) 0:00:19.185 ***** 2026-01-21 11:07:35.552253 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.706) 0:00:19.184 ***** 2026-01-21 11:07:35.552264 | controller | skipping: [controller] 2026-01-21 11:07:35.590868 | controller | 2026-01-21 11:07:35.590963 | controller | TASK [repo_setup : Rename component repo path={{ cifmw_repo_setup_output }}/{{ cifmw_repo_setup_component_name }}_{{ cifmw_repo_setup_component_promotion_tag }}_delorean.repo, regexp=delorean-component-{{ cifmw_repo_setup_component_name }}, replace={{ cifmw_repo_setup_component_name }}-{{ cifmw_repo_setup_component_promotion_tag }}] *** 2026-01-21 11:07:35.590980 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.037) 0:00:19.223 ***** 2026-01-21 11:07:35.590988 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.037) 0:00:19.222 ***** 2026-01-21 11:07:35.591004 | controller | skipping: [controller] 2026-01-21 11:07:35.634067 | controller | 2026-01-21 11:07:35.634147 | controller | TASK [repo_setup : Disable component repo in current-podified dlrn repo path={{ cifmw_repo_setup_output }}/delorean.repo, section=delorean-component-{{ cifmw_repo_setup_component_name }}, option=enabled, value=0, mode=0644] *** 2026-01-21 11:07:35.634157 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.038) 0:00:19.262 ***** 2026-01-21 11:07:35.634166 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.038) 0:00:19.260 ***** 2026-01-21 11:07:35.634182 | controller | skipping: [controller] 2026-01-21 11:07:36.242122 | controller | 2026-01-21 11:07:36.242181 | controller | TASK [repo_setup : Run repo-setup-get-hash _raw_params={{ cifmw_repo_setup_venv }}/bin/repo-setup-get-hash --dlrn-url {{ cifmw_repo_setup_dlrn_uri[:-1] }} --os-version {{ cifmw_repo_setup_os_release }}{{ cifmw_repo_setup_dist_major_version }} --release {{ cifmw_repo_setup_branch }} {% if cifmw_repo_setup_component_name | length > 0 -%} --component {{ cifmw_repo_setup_component_name }} --tag {{ cifmw_repo_setup_component_promotion_tag }} {% else -%} --tag {{cifmw_repo_setup_promotion }} {% endif -%} {% if (cifmw_repo_setup_dlrn_hash_tag | length > 0) and (cifmw_repo_setup_component_name | length <= 0) -%} --dlrn-hash-tag {{ cifmw_repo_setup_dlrn_hash_tag }} {% endif -%} --json] *** 2026-01-21 11:07:36.242194 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.043) 0:00:19.305 ***** 2026-01-21 11:07:36.242210 | controller | Wednesday 21 January 2026 11:07:35 +0000 (0:00:00.043) 0:00:19.304 ***** 2026-01-21 11:07:36.242231 | controller | changed: [controller] 2026-01-21 11:07:37.331817 | controller | 2026-01-21 11:07:37.331899 | controller | TASK [repo_setup : Dump full hash in delorean.repo.md5 file content={{ _repo_setup_json['full_hash'] }} 2026-01-21 11:07:37.331916 | controller | , dest={{ cifmw_repo_setup_basedir }}/artifacts/repositories/delorean.repo.md5, mode=0644] *** 2026-01-21 11:07:37.331935 | controller | Wednesday 21 January 2026 11:07:36 +0000 (0:00:00.607) 0:00:19.913 ***** 2026-01-21 11:07:37.331943 | controller | Wednesday 21 January 2026 11:07:36 +0000 (0:00:00.608) 0:00:19.912 ***** 2026-01-21 11:07:37.331957 | controller | changed: [controller] 2026-01-21 11:07:37.359374 | controller | 2026-01-21 11:07:37.359630 | controller | TASK [repo_setup : Dump current-podified hash url={{ cifmw_repo_setup_dlrn_uri }}/{{ cifmw_repo_setup_os_release }}{{ cifmw_repo_setup_dist_major_version }}-{{ cifmw_repo_setup_branch }}/current-podified/delorean.repo.md5, dest={{ cifmw_repo_setup_basedir }}/artifacts/repositories/delorean.repo.md5, mode=0644] *** 2026-01-21 11:07:37.359641 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:01.089) 0:00:21.003 ***** 2026-01-21 11:07:37.359655 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:01.089) 0:00:21.001 ***** 2026-01-21 11:07:37.359667 | controller | skipping: [controller] 2026-01-21 11:07:37.384275 | controller | 2026-01-21 11:07:37.384316 | controller | TASK [repo_setup : Slurp current podified hash src={{ cifmw_repo_setup_basedir }}/artifacts/repositories/delorean.repo.md5] *** 2026-01-21 11:07:37.384324 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.027) 0:00:21.030 ***** 2026-01-21 11:07:37.384332 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.027) 0:00:21.029 ***** 2026-01-21 11:07:37.384343 | controller | skipping: [controller] 2026-01-21 11:07:37.409753 | controller | 2026-01-21 11:07:37.409807 | controller | TASK [repo_setup : Update the value of full_hash _repo_setup_json={{ _repo_setup_json | combine({'full_hash': _hash}, recursive=true) }}] *** 2026-01-21 11:07:37.409832 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.055 ***** 2026-01-21 11:07:37.409842 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.054 ***** 2026-01-21 11:07:37.409857 | controller | skipping: [controller] 2026-01-21 11:07:37.446977 | controller | 2026-01-21 11:07:37.447017 | controller | TASK [repo_setup : Export hashes facts for further use cifmw_repo_setup_full_hash={{ _repo_setup_json['full_hash'] }}, cifmw_repo_setup_commit_hash={{ _repo_setup_json['commit_hash'] }}, cifmw_repo_setup_distro_hash={{ _repo_setup_json['distro_hash'] }}, cifmw_repo_setup_extended_hash={{ _repo_setup_json['extended_hash'] }}, cifmw_repo_setup_dlrn_api_url={{ _repo_setup_json['dlrn_api_url'] }}, cifmw_repo_setup_dlrn_url={{ _repo_setup_json['dlrn_url'] }}, cifmw_repo_setup_release={{ _repo_setup_json['release'] }}, cacheable=True] *** 2026-01-21 11:07:37.447027 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.081 ***** 2026-01-21 11:07:37.447034 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.079 ***** 2026-01-21 11:07:37.447047 | controller | ok: [controller] 2026-01-21 11:07:37.470320 | controller | 2026-01-21 11:07:37.470360 | controller | TASK [repo_setup : Create download directory path={{ cifmw_repo_setup_rhos_release_path }}, state=directory, mode=0755] *** 2026-01-21 11:07:37.470369 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.037) 0:00:21.118 ***** 2026-01-21 11:07:37.470376 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.037) 0:00:21.117 ***** 2026-01-21 11:07:37.470388 | controller | skipping: [controller] 2026-01-21 11:07:37.492860 | controller | 2026-01-21 11:07:37.492900 | controller | TASK [repo_setup : Print the URL to request msg={{ cifmw_repo_setup_rhos_release_rpm }}] *** 2026-01-21 11:07:37.492909 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.023) 0:00:21.141 ***** 2026-01-21 11:07:37.492916 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.023) 0:00:21.140 ***** 2026-01-21 11:07:37.492928 | controller | skipping: [controller] 2026-01-21 11:07:37.515971 | controller | 2026-01-21 11:07:37.516013 | controller | TASK [Download the RPM name=krb_request] *************************************** 2026-01-21 11:07:37.516022 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.022) 0:00:21.164 ***** 2026-01-21 11:07:37.516029 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.022) 0:00:21.163 ***** 2026-01-21 11:07:37.516041 | controller | skipping: [controller] 2026-01-21 11:07:37.539623 | controller | 2026-01-21 11:07:37.539665 | controller | TASK [repo_setup : Install RHOS Release tool name={{ 2026-01-21 11:07:37.539674 | controller | cifmw_repo_setup_rhos_release_rpm 2026-01-21 11:07:37.539682 | controller | if cifmw_repo_setup_rhos_release_rpm is not url 2026-01-21 11:07:37.539688 | controller | else cifmw_krb_request_out.path 2026-01-21 11:07:37.539695 | controller | }}, state=present, disable_gpg_check={{ cifmw_repo_setup_rhos_release_gpg_check | bool }}] *** 2026-01-21 11:07:37.539702 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.023) 0:00:21.187 ***** 2026-01-21 11:07:37.539709 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.023) 0:00:21.186 ***** 2026-01-21 11:07:37.539744 | controller | skipping: [controller] 2026-01-21 11:07:37.564956 | controller | 2026-01-21 11:07:37.565000 | controller | TASK [repo_setup : Get rhos-release tool version _raw_params=rhos-release --version] *** 2026-01-21 11:07:37.565017 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.023) 0:00:21.211 ***** 2026-01-21 11:07:37.565025 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.023) 0:00:21.209 ***** 2026-01-21 11:07:37.565037 | controller | skipping: [controller] 2026-01-21 11:07:37.590823 | controller | 2026-01-21 11:07:37.590866 | controller | TASK [repo_setup : Print rhos-release tool version msg={{ rr_version.stdout }}] *** 2026-01-21 11:07:37.590881 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.236 ***** 2026-01-21 11:07:37.590888 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.235 ***** 2026-01-21 11:07:37.590901 | controller | skipping: [controller] 2026-01-21 11:07:37.616354 | controller | 2026-01-21 11:07:37.616431 | controller | TASK [repo_setup : Generate repos using rhos-release {{ cifmw_repo_setup_rhos_release_args }} _raw_params=rhos-release {{ cifmw_repo_setup_rhos_release_args }} \ 2026-01-21 11:07:37.616441 | controller | -t {{ cifmw_repo_setup_output }}] *** 2026-01-21 11:07:37.616448 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.026) 0:00:21.262 ***** 2026-01-21 11:07:37.616456 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.026) 0:00:21.261 ***** 2026-01-21 11:07:37.616472 | controller | skipping: [controller] 2026-01-21 11:07:37.941246 | controller | 2026-01-21 11:07:37.941308 | controller | TASK [repo_setup : Check for /etc/ci/mirror_info.sh path=/etc/ci/mirror_info.sh] *** 2026-01-21 11:07:37.941325 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.287 ***** 2026-01-21 11:07:37.941332 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.025) 0:00:21.286 ***** 2026-01-21 11:07:37.941345 | controller | ok: [controller] 2026-01-21 11:07:38.202402 | controller | 2026-01-21 11:07:38.202568 | controller | TASK [repo_setup : Use RDO proxy mirrors chdir={{ cifmw_repo_setup_output }}, _raw_params=set -o pipefail 2026-01-21 11:07:38.202587 | controller | source /etc/ci/mirror_info.sh 2026-01-21 11:07:38.202597 | controller | sed -i -e "s|https://trunk.rdoproject.org|$NODEPOOL_RDO_PROXY|g" *.repo 2026-01-21 11:07:38.202606 | controller | ] *** 2026-01-21 11:07:38.202615 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.324) 0:00:21.612 ***** 2026-01-21 11:07:38.202624 | controller | Wednesday 21 January 2026 11:07:37 +0000 (0:00:00.324) 0:00:21.611 ***** 2026-01-21 11:07:38.202654 | controller | changed: [controller] 2026-01-21 11:07:38.495339 | controller | 2026-01-21 11:07:38.495423 | controller | TASK [repo_setup : Use RDO CentOS mirrors (remove CentOS 10 conditional when Nodepool mirrors exist) chdir={{ cifmw_repo_setup_output }}, _raw_params=set -o pipefail 2026-01-21 11:07:38.495437 | controller | source /etc/ci/mirror_info.sh 2026-01-21 11:07:38.495446 | controller | sed -i -e "s|http://mirror.stream.centos.org|$NODEPOOL_CENTOS_MIRROR|g" *.repo 2026-01-21 11:07:38.495454 | controller | ] *** 2026-01-21 11:07:38.495461 | controller | Wednesday 21 January 2026 11:07:38 +0000 (0:00:00.261) 0:00:21.873 ***** 2026-01-21 11:07:38.495467 | controller | Wednesday 21 January 2026 11:07:38 +0000 (0:00:00.261) 0:00:21.872 ***** 2026-01-21 11:07:38.495480 | controller | changed: [controller] 2026-01-21 11:07:39.310591 | controller | 2026-01-21 11:07:39.310680 | controller | TASK [repo_setup : Check for gating.repo file on content provider url=http://{{ content_provider_registry_ip }}:8766/gating.repo] *** 2026-01-21 11:07:39.310694 | controller | Wednesday 21 January 2026 11:07:38 +0000 (0:00:00.292) 0:00:22.166 ***** 2026-01-21 11:07:39.310704 | controller | Wednesday 21 January 2026 11:07:38 +0000 (0:00:00.292) 0:00:22.165 ***** 2026-01-21 11:07:39.310734 | controller | task path: /home/zuul/src/github.com/openstack-k8s-operators/ci-framework/roles/repo_setup/tasks/populate_gating_repo.yml:2 2026-01-21 11:07:39.355480 | controller | fatal: [controller]: FAILED! => 2026-01-21 11:07:39.355521 | controller | changed: false 2026-01-21 11:07:39.355529 | controller | elapsed: 0 2026-01-21 11:07:39.355537 | controller | msg: 'Status code was -1 and not [200]: Request failed: ' 2026-01-21 11:07:39.355550 | controller | redirected: false 2026-01-21 11:07:39.355556 | controller | status: -1 2026-01-21 11:07:39.355563 | controller | url: http://38.102.83.182:8766/gating.repo 2026-01-21 11:07:39.355577 | controller | ...ignoring 2026-01-21 11:07:39.355583 | controller | 2026-01-21 11:07:39.355590 | controller | TASK [repo_setup : Populate gating repo from content provider ip content=[gating-repo] 2026-01-21 11:07:39.355597 | controller | baseurl=http://{{ content_provider_registry_ip }}:8766/ 2026-01-21 11:07:39.355603 | controller | enabled=1 2026-01-21 11:07:39.355610 | controller | gpgcheck=0 2026-01-21 11:07:39.355616 | controller | priority=1 2026-01-21 11:07:39.355622 | controller | , dest={{ cifmw_repo_setup_output }}/gating.repo, mode=0644] *** 2026-01-21 11:07:39.355629 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.815) 0:00:22.982 ***** 2026-01-21 11:07:39.355635 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.815) 0:00:22.980 ***** 2026-01-21 11:07:39.355648 | controller | skipping: [controller] 2026-01-21 11:07:39.418222 | controller | 2026-01-21 11:07:39.418312 | controller | TASK [repo_setup : Check for DLRN repo at the destination path={{ cifmw_repo_setup_output }}/delorean.repo] *** 2026-01-21 11:07:39.418323 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.045) 0:00:23.027 ***** 2026-01-21 11:07:39.418332 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.045) 0:00:23.025 ***** 2026-01-21 11:07:39.418353 | controller | skipping: [controller] 2026-01-21 11:07:39.462691 | controller | 2026-01-21 11:07:39.462764 | controller | TASK [repo_setup : Lower the priority of DLRN repos to allow installation from gating repo path={{ cifmw_repo_setup_output }}/delorean.repo, regexp=priority=1, replace=priority=20] *** 2026-01-21 11:07:39.462776 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.061) 0:00:23.088 ***** 2026-01-21 11:07:39.462784 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.061) 0:00:23.087 ***** 2026-01-21 11:07:39.462798 | controller | skipping: [controller] 2026-01-21 11:07:39.507766 | controller | 2026-01-21 11:07:39.507819 | controller | TASK [repo_setup : Check for DLRN component repo path={{ cifmw_repo_setup_output }}/{{ _comp_repo }}] *** 2026-01-21 11:07:39.507830 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.045) 0:00:23.134 ***** 2026-01-21 11:07:39.507838 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.045) 0:00:23.132 ***** 2026-01-21 11:07:39.507853 | controller | skipping: [controller] 2026-01-21 11:07:39.545365 | controller | 2026-01-21 11:07:39.545408 | controller | TASK [repo_setup : Lower the priority of componennt repos to allow installation from gating repo path={{ cifmw_repo_setup_output }}//{{ _comp_repo }}, regexp=priority=1, replace=priority=2] *** 2026-01-21 11:07:39.545417 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.045) 0:00:23.179 ***** 2026-01-21 11:07:39.545424 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.045) 0:00:23.177 ***** 2026-01-21 11:07:39.545437 | controller | skipping: [controller] 2026-01-21 11:07:39.976924 | controller | 2026-01-21 11:07:39.976966 | controller | TASK [repo_setup : Find existing repos from /etc/yum.repos.d directory paths=/etc/yum.repos.d/, patterns=*.repo, recurse=False] *** 2026-01-21 11:07:39.976985 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.037) 0:00:23.216 ***** 2026-01-21 11:07:39.976993 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.037) 0:00:23.215 ***** 2026-01-21 11:07:39.977005 | controller | ok: [controller] 2026-01-21 11:07:40.551561 | controller | 2026-01-21 11:07:40.551606 | controller | TASK [repo_setup : Remove existing repos from /etc/yum.repos.d directory path={{ item }}, state=absent] *** 2026-01-21 11:07:40.551617 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.431) 0:00:23.648 ***** 2026-01-21 11:07:40.551625 | controller | Wednesday 21 January 2026 11:07:39 +0000 (0:00:00.431) 0:00:23.647 ***** 2026-01-21 11:07:40.551640 | controller | changed: [controller] => (item=/etc/yum.repos.d/centos-addons.repo) 2026-01-21 11:07:41.229841 | controller | changed: [controller] => (item=/etc/yum.repos.d/centos.repo) 2026-01-21 11:07:41.229924 | controller | 2026-01-21 11:07:41.229933 | controller | TASK [repo_setup : Cleanup existing metadata _raw_params=dnf clean metadata] *** 2026-01-21 11:07:41.229940 | controller | Wednesday 21 January 2026 11:07:40 +0000 (0:00:00.574) 0:00:24.222 ***** 2026-01-21 11:07:41.229957 | controller | Wednesday 21 January 2026 11:07:40 +0000 (0:00:00.574) 0:00:24.221 ***** 2026-01-21 11:07:41.229980 | controller | changed: [controller] 2026-01-21 11:07:41.690887 | controller | 2026-01-21 11:07:41.691019 | controller | TASK [repo_setup : Copy generated repos to /etc/yum.repos.d directory mode=0755, remote_src=True, src={{ cifmw_repo_setup_output }}/, dest=/etc/yum.repos.d] *** 2026-01-21 11:07:41.691037 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.678) 0:00:24.901 ***** 2026-01-21 11:07:41.691052 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.677) 0:00:24.899 ***** 2026-01-21 11:07:41.691068 | controller | changed: [controller] 2026-01-21 11:07:41.751340 | controller | 2026-01-21 11:07:41.751381 | controller | TASK [ci_setup : Gather variables for each operating system _raw_params={{ item }}] *** 2026-01-21 11:07:41.751390 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.461) 0:00:25.362 ***** 2026-01-21 11:07:41.751397 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.461) 0:00:25.360 ***** 2026-01-21 11:07:41.751408 | controller | ok: [controller] => (item=/home/zuul/src/github.com/openstack-k8s-operators/ci-framework/roles/ci_setup/vars/redhat.yml) 2026-01-21 11:07:41.780046 | controller | 2026-01-21 11:07:41.780088 | controller | TASK [ci_setup : List packages to install var=cifmw_ci_setup_packages] ********* 2026-01-21 11:07:41.780097 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.060) 0:00:25.423 ***** 2026-01-21 11:07:41.780104 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.060) 0:00:25.421 ***** 2026-01-21 11:07:41.780115 | controller | ok: [controller] => 2026-01-21 11:08:19.544108 | controller | cifmw_ci_setup_packages: 2026-01-21 11:08:19.544292 | controller | - bash-completion 2026-01-21 11:08:19.544307 | controller | - ca-certificates 2026-01-21 11:08:19.544316 | controller | - git-core 2026-01-21 11:08:19.544325 | controller | - make 2026-01-21 11:08:19.544334 | controller | - tar 2026-01-21 11:08:19.544344 | controller | - tmux 2026-01-21 11:08:19.544353 | controller | - python3-pip 2026-01-21 11:08:19.544361 | controller | 2026-01-21 11:08:19.544370 | controller | TASK [ci_setup : Install needed packages name={{ cifmw_ci_setup_packages }}, state=latest] *** 2026-01-21 11:08:19.544380 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.028) 0:00:25.451 ***** 2026-01-21 11:08:19.544397 | controller | Wednesday 21 January 2026 11:07:41 +0000 (0:00:00.028) 0:00:25.450 ***** 2026-01-21 11:08:19.544423 | controller | changed: [controller] 2026-01-21 11:08:19.821566 | controller | 2026-01-21 11:08:19.821615 | controller | TASK [ci_setup : Gather version of openshift client _raw_params=oc version --client -o yaml] *** 2026-01-21 11:08:19.821627 | controller | Wednesday 21 January 2026 11:08:19 +0000 (0:00:37.762) 0:01:03.214 ***** 2026-01-21 11:08:19.821635 | controller | Wednesday 21 January 2026 11:08:19 +0000 (0:00:37.763) 0:01:03.213 ***** 2026-01-21 11:08:19.821650 | controller | ok: [controller] 2026-01-21 11:08:20.101406 | controller | 2026-01-21 11:08:20.101475 | controller | TASK [ci_setup : Ensure openshift client install path is present path={{ cifmw_ci_setup_oc_install_path }}, state=directory, mode=0755] *** 2026-01-21 11:08:20.101511 | controller | Wednesday 21 January 2026 11:08:19 +0000 (0:00:00.278) 0:01:03.493 ***** 2026-01-21 11:08:20.101527 | controller | Wednesday 21 January 2026 11:08:19 +0000 (0:00:00.278) 0:01:03.491 ***** 2026-01-21 11:08:20.101551 | controller | changed: [controller] 2026-01-21 11:08:26.256742 | controller | 2026-01-21 11:08:26.256929 | controller | TASK [ci_setup : Install openshift client src={{ cifmw_ci_setup_openshift_client_download_uri }}/{{ cifmw_ci_setup_openshift_client_version }}/openshift-client-linux.tar.gz, dest={{ cifmw_ci_setup_oc_install_path }}, remote_src=True, mode=0755, creates={{ cifmw_ci_setup_oc_install_path }}/oc] *** 2026-01-21 11:08:26.256951 | controller | Wednesday 21 January 2026 11:08:20 +0000 (0:00:00.278) 0:01:03.771 ***** 2026-01-21 11:08:26.256968 | controller | Wednesday 21 January 2026 11:08:20 +0000 (0:00:00.278) 0:01:03.770 ***** 2026-01-21 11:08:26.256990 | controller | changed: [controller] 2026-01-21 11:08:26.299837 | controller | 2026-01-21 11:08:26.299901 | controller | TASK [ci_setup : Add the OC path to cifmw_path if needed cifmw_path={{ cifmw_ci_setup_oc_install_path }}:{{ ansible_env.PATH }}, cacheable=True] *** 2026-01-21 11:08:26.299923 | controller | Wednesday 21 January 2026 11:08:26 +0000 (0:00:06.154) 0:01:09.926 ***** 2026-01-21 11:08:26.299930 | controller | Wednesday 21 January 2026 11:08:26 +0000 (0:00:06.154) 0:01:09.925 ***** 2026-01-21 11:08:26.299958 | controller | skipping: [controller] 2026-01-21 11:08:26.704174 | controller | 2026-01-21 11:08:26.704222 | controller | TASK [ci_setup : Create completion file] *************************************** 2026-01-21 11:08:26.704233 | controller | Wednesday 21 January 2026 11:08:26 +0000 (0:00:00.044) 0:01:09.971 ***** 2026-01-21 11:08:26.704241 | controller | Wednesday 21 January 2026 11:08:26 +0000 (0:00:00.044) 0:01:09.969 ***** 2026-01-21 11:08:26.704255 | controller | changed: [controller] 2026-01-21 11:08:27.074033 | controller | 2026-01-21 11:08:27.074084 | controller | TASK [ci_setup : Source completion from within .bashrc create=True, mode=0644, path={{ ansible_user_dir }}/.bashrc, block=if [ -f ~/.oc_completion ]; then 2026-01-21 11:08:27.074105 | controller | source ~/.oc_completion 2026-01-21 11:08:27.074115 | controller | fi] *** 2026-01-21 11:08:27.074123 | controller | Wednesday 21 January 2026 11:08:26 +0000 (0:00:00.404) 0:01:10.375 ***** 2026-01-21 11:08:27.074132 | controller | Wednesday 21 January 2026 11:08:26 +0000 (0:00:00.404) 0:01:10.374 ***** 2026-01-21 11:08:27.074146 | controller | changed: [controller] 2026-01-21 11:08:27.095805 | controller | 2026-01-21 11:08:27.095851 | controller | TASK [ci_setup : Check rhsm status _raw_params=subscription-manager status] **** 2026-01-21 11:08:27.095861 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.369) 0:01:10.745 ***** 2026-01-21 11:08:27.095869 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.370) 0:01:10.744 ***** 2026-01-21 11:08:27.095883 | controller | skipping: [controller] 2026-01-21 11:08:27.117838 | controller | 2026-01-21 11:08:27.117885 | controller | TASK [ci_setup : Gather the repos to be enabled _repos={{ 2026-01-21 11:08:27.117895 | controller | cifmw_ci_setup_rhel_rhsm_default_repos + 2026-01-21 11:08:27.117904 | controller | (cifmw_ci_setup_rhel_rhsm_extra_repos | default([])) 2026-01-21 11:08:27.117912 | controller | }}] *** 2026-01-21 11:08:27.117920 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.021) 0:01:10.767 ***** 2026-01-21 11:08:27.117928 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.021) 0:01:10.766 ***** 2026-01-21 11:08:27.117943 | controller | skipping: [controller] 2026-01-21 11:08:27.140377 | controller | 2026-01-21 11:08:27.140450 | controller | TASK [ci_setup : Enabling the required repositories. name={{ item }}, state={{ rhsm_repo_state | default('enabled') }}] *** 2026-01-21 11:08:27.140459 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.021) 0:01:10.789 ***** 2026-01-21 11:08:27.140478 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.021) 0:01:10.787 ***** 2026-01-21 11:08:27.140496 | controller | skipping: [controller] 2026-01-21 11:08:27.164074 | controller | 2026-01-21 11:08:27.164159 | controller | TASK [ci_setup : Get current /etc/redhat-release _raw_params=cat /etc/redhat-release] *** 2026-01-21 11:08:27.164169 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.022) 0:01:10.811 ***** 2026-01-21 11:08:27.164178 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.022) 0:01:10.810 ***** 2026-01-21 11:08:27.164201 | controller | skipping: [controller] 2026-01-21 11:08:27.190818 | controller | 2026-01-21 11:08:27.190900 | controller | TASK [ci_setup : Print current /etc/redhat-release msg={{ _current_rh_release.stdout }}] *** 2026-01-21 11:08:27.190911 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.023) 0:01:10.835 ***** 2026-01-21 11:08:27.190920 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.023) 0:01:10.834 ***** 2026-01-21 11:08:27.190942 | controller | skipping: [controller] 2026-01-21 11:08:27.222092 | controller | 2026-01-21 11:08:27.222171 | controller | TASK [ci_setup : Ensure the repos are enabled in the system using yum name={{ item.name }}, baseurl={{ item.baseurl }}, description={{ item.description | default(item.name) }}, gpgcheck={{ item.gpgcheck | default(false) }}, enabled=True, state={{ yum_repo_state | default('present') }}] *** 2026-01-21 11:08:27.222183 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.026) 0:01:10.862 ***** 2026-01-21 11:08:27.222192 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.026) 0:01:10.860 ***** 2026-01-21 11:08:27.222213 | controller | skipping: [controller] 2026-01-21 11:08:28.605084 | controller | 2026-01-21 11:08:28.605129 | controller | TASK [ci_setup : Manage directories path={{ item }}, state={{ directory_state }}, mode=0755, owner={{ ansible_user_id }}, group={{ ansible_user_id }}] *** 2026-01-21 11:08:28.605152 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.031) 0:01:10.893 ***** 2026-01-21 11:08:28.605161 | controller | Wednesday 21 January 2026 11:08:27 +0000 (0:00:00.031) 0:01:10.892 ***** 2026-01-21 11:08:28.605172 | controller | changed: [controller] => (item=/home/zuul/ci-framework-data/artifacts/manifests/openstack/cr) 2026-01-21 11:08:28.751809 | controller | changed: [controller] => (item=/home/zuul/ci-framework-data/logs) 2026-01-21 11:08:28.751847 | controller | ok: [controller] => (item=/home/zuul/ci-framework-data/tmp) 2026-01-21 11:08:28.751861 | controller | changed: [controller] => (item=/home/zuul/ci-framework-data/volumes) 2026-01-21 11:08:28.751869 | controller | ok: [controller] => (item=/home/zuul/ci-framework-data/artifacts/parameters) 2026-01-21 11:08:28.751876 | controller | 2026-01-21 11:08:28.751882 | controller | TASK [Prepare install_yamls make targets name=install_yamls, apply={'tags': ['bootstrap']}] *** 2026-01-21 11:08:28.751889 | controller | Wednesday 21 January 2026 11:08:28 +0000 (0:00:01.382) 0:01:12.276 ***** 2026-01-21 11:08:28.751896 | controller | Wednesday 21 January 2026 11:08:28 +0000 (0:00:01.382) 0:01:12.275 ***** 2026-01-21 11:08:28.751907 | controller | 2026-01-21 11:08:29.492399 | controller | TASK [install_yamls : Ensure directories exist path={{ item }}, state=directory, mode=0755] *** 2026-01-21 11:08:29.492450 | controller | Wednesday 21 January 2026 11:08:28 +0000 (0:00:00.146) 0:01:12.423 ***** 2026-01-21 11:08:29.492459 | controller | Wednesday 21 January 2026 11:08:28 +0000 (0:00:00.147) 0:01:12.422 ***** 2026-01-21 11:08:29.492471 | controller | ok: [controller] => (item=/home/zuul/ci-framework-data/artifacts) 2026-01-21 11:08:29.535963 | controller | changed: [controller] => (item=/home/zuul/ci-framework-data/artifacts/roles/install_yamls_makes/tasks) 2026-01-21 11:08:29.536006 | controller | ok: [controller] => (item=/home/zuul/ci-framework-data/artifacts/parameters) 2026-01-21 11:08:29.536015 | controller | 2026-01-21 11:08:29.536022 | controller | TASK [Create variables with local repos based on Zuul items name=install_yamls, tasks_from=zuul_set_operators_repo.yml] *** 2026-01-21 11:08:29.536029 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.740) 0:01:13.163 ***** 2026-01-21 11:08:29.536035 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.740) 0:01:13.162 ***** 2026-01-21 11:08:29.536047 | controller | 2026-01-21 11:08:29.599647 | controller | TASK [install_yamls : Set fact with local repos based on Zuul items cifmw_install_yamls_operators_repo={{ cifmw_install_yamls_operators_repo | default({}) | combine(_repo_operator_info | items2dict) }}] *** 2026-01-21 11:08:29.599695 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.043) 0:01:13.207 ***** 2026-01-21 11:08:29.599706 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.043) 0:01:13.206 ***** 2026-01-21 11:08:29.599742 | controller | ok: [controller] => (item={'branch': 'main', 'change': '320', 'change_url': 'https://github.com/openstack-k8s-operators/watcher-operator/pull/320', 'commit_id': '111af6c57cd1be8032328ced6eb7a058a201c4b1', 'patchset': '111af6c57cd1be8032328ced6eb7a058a201c4b1', 'project': {'canonical_hostname': 'github.com', 'canonical_name': 'github.com/openstack-k8s-operators/watcher-operator', 'name': 'openstack-k8s-operators/watcher-operator', 'short_name': 'watcher-operator', 'src_dir': 'src/github.com/openstack-k8s-operators/watcher-operator'}, 'topic': None}) 2026-01-21 11:08:29.664898 | controller | 2026-01-21 11:08:29.664942 | controller | TASK [install_yamls : Print helpful data for debugging msg=_repo_operator_name: {{ _repo_operator_name }} 2026-01-21 11:08:29.664952 | controller | _repo_operator_info: {{ _repo_operator_info }} 2026-01-21 11:08:29.664961 | controller | cifmw_install_yamls_operators_repo: {{ cifmw_install_yamls_operators_repo }} 2026-01-21 11:08:29.664970 | controller | ] *** 2026-01-21 11:08:29.664978 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.063) 0:01:13.271 ***** 2026-01-21 11:08:29.664987 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.063) 0:01:13.269 ***** 2026-01-21 11:08:29.665003 | controller | ok: [controller] => (item={'branch': 'main', 'change': '320', 'change_url': 'https://github.com/openstack-k8s-operators/watcher-operator/pull/320', 'commit_id': '111af6c57cd1be8032328ced6eb7a058a201c4b1', 'patchset': '111af6c57cd1be8032328ced6eb7a058a201c4b1', 'project': {'canonical_hostname': 'github.com', 'canonical_name': 'github.com/openstack-k8s-operators/watcher-operator', 'name': 'openstack-k8s-operators/watcher-operator', 'short_name': 'watcher-operator', 'src_dir': 'src/github.com/openstack-k8s-operators/watcher-operator'}, 'topic': None}) => 2026-01-21 11:08:29.708091 | controller | msg: | 2026-01-21 11:08:29.708130 | controller | _repo_operator_name: watcher 2026-01-21 11:08:29.708149 | controller | _repo_operator_info: [{'key': 'WATCHER_REPO', 'value': '/home/zuul/src/github.com/openstack-k8s-operators/watcher-operator'}, {'key': 'WATCHER_BRANCH', 'value': ''}] 2026-01-21 11:08:29.708157 | controller | cifmw_install_yamls_operators_repo: {'WATCHER_REPO': '/home/zuul/src/github.com/openstack-k8s-operators/watcher-operator', 'WATCHER_BRANCH': ''} 2026-01-21 11:08:29.708164 | controller | 2026-01-21 11:08:29.708171 | controller | TASK [Customize install_yamls devsetup vars if needed name=install_yamls, tasks_from=customize_devsetup_vars.yml] *** 2026-01-21 11:08:29.708178 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.065) 0:01:13.336 ***** 2026-01-21 11:08:29.708185 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.065) 0:01:13.335 ***** 2026-01-21 11:08:29.708197 | controller | 2026-01-21 11:08:29.735379 | controller | TASK [install_yamls : Update opm_version in install_yamls devsetup/vars/default.yaml path={{ cifmw_install_yamls_repo }}/devsetup/vars/default.yaml, regexp=^opm_version:, line=opm_version: {{ cifmw_install_yamls_opm_version }}, state=present] *** 2026-01-21 11:08:29.735432 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.043) 0:01:13.379 ***** 2026-01-21 11:08:29.735441 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.043) 0:01:13.378 ***** 2026-01-21 11:08:29.735453 | controller | skipping: [controller] 2026-01-21 11:08:29.771239 | controller | 2026-01-21 11:08:29.771281 | controller | TASK [install_yamls : Update sdk_version in install_yamls devsetup/vars/default.yaml path={{ cifmw_install_yamls_repo }}/devsetup/vars/default.yaml, regexp=^sdk_version:, line=sdk_version: {{ cifmw_install_yamls_sdk_version }}, state=present] *** 2026-01-21 11:08:29.771291 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.027) 0:01:13.407 ***** 2026-01-21 11:08:29.771299 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.027) 0:01:13.405 ***** 2026-01-21 11:08:29.771311 | controller | skipping: [controller] 2026-01-21 11:08:29.797513 | controller | 2026-01-21 11:08:29.797554 | controller | TASK [install_yamls : Update go_version in install_yamls devsetup/vars/default.yaml path={{ cifmw_install_yamls_repo }}/devsetup/vars/default.yaml, regexp=^go_version:, line=go_version: {{ cifmw_install_yamls_go_version }}, state=present] *** 2026-01-21 11:08:29.797564 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.035) 0:01:13.442 ***** 2026-01-21 11:08:29.797571 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.035) 0:01:13.441 ***** 2026-01-21 11:08:29.797583 | controller | skipping: [controller] 2026-01-21 11:08:29.843215 | controller | 2026-01-21 11:08:29.843262 | controller | TASK [install_yamls : Update kustomize_version in install_yamls devsetup/vars/default.yaml path={{ cifmw_install_yamls_repo }}/devsetup/vars/default.yaml, regexp=^kustomize_version:, line=kustomize_version: {{ cifmw_install_yamls_kustomize_version }}, state=present] *** 2026-01-21 11:08:29.843271 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.026) 0:01:13.469 ***** 2026-01-21 11:08:29.843285 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.026) 0:01:13.467 ***** 2026-01-21 11:08:29.843298 | controller | skipping: [controller] 2026-01-21 11:08:29.932625 | controller | 2026-01-21 11:08:29.932667 | controller | TASK [install_yamls : Compute the cifmw_install_yamls_vars final value _install_yamls_override_vars={{ 2026-01-21 11:08:29.932676 | controller | _install_yamls_override_vars | default({}) | 2026-01-21 11:08:29.932683 | controller | combine(item, recursive=True) 2026-01-21 11:08:29.932690 | controller | }}] *** 2026-01-21 11:08:29.932706 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.045) 0:01:13.514 ***** 2026-01-21 11:08:29.932727 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.045) 0:01:13.513 ***** 2026-01-21 11:08:29.932743 | controller | ok: [controller] => (item={'BMO_SETUP': False}) 2026-01-21 11:08:29.978849 | controller | 2026-01-21 11:08:29.978886 | controller | TASK [install_yamls : Set environment override cifmw_install_yamls_environment fact cifmw_install_yamls_environment={{ 2026-01-21 11:08:29.978895 | controller | _install_yamls_override_vars.keys() | 2026-01-21 11:08:29.978902 | controller | map('upper') | 2026-01-21 11:08:29.978909 | controller | zip(_install_yamls_override_vars.values()) | 2026-01-21 11:08:29.978915 | controller | items2dict(key_name=0, value_name=1) | 2026-01-21 11:08:29.978922 | controller | combine({ 2026-01-21 11:08:29.978929 | controller | 'OUT': cifmw_install_yamls_manifests_dir, 2026-01-21 11:08:29.978942 | controller | 'OUTPUT_DIR': cifmw_install_yamls_edpm_dir, 2026-01-21 11:08:29.978948 | controller | 'CHECKOUT_FROM_OPENSTACK_REF': cifmw_install_yamls_checkout_openstack_ref, 2026-01-21 11:08:29.978955 | controller | 'OPENSTACK_K8S_BRANCH': (zuul is defined and not zuul.branch |regex_search('master|antelope|rhos')) | ternary(zuul.branch, 'main') 2026-01-21 11:08:29.978962 | controller | }) | 2026-01-21 11:08:29.978969 | controller | combine(install_yamls_operators_repos) 2026-01-21 11:08:29.978976 | controller | }}, cacheable=True] *** 2026-01-21 11:08:29.978982 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.089) 0:01:13.604 ***** 2026-01-21 11:08:29.978989 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.089) 0:01:13.602 ***** 2026-01-21 11:08:29.979000 | controller | ok: [controller] 2026-01-21 11:08:30.643872 | controller | 2026-01-21 11:08:30.643926 | controller | TASK [install_yamls : Get environment structure base_path={{ cifmw_install_yamls_repo }}] *** 2026-01-21 11:08:30.643936 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.046) 0:01:13.650 ***** 2026-01-21 11:08:30.643943 | controller | Wednesday 21 January 2026 11:08:29 +0000 (0:00:00.046) 0:01:13.649 ***** 2026-01-21 11:08:30.643961 | controller | ok: [controller] 2026-01-21 11:08:30.901739 | controller | 2026-01-21 11:08:30.901820 | controller | TASK [install_yamls : Ensure Output directory exists path={{ cifmw_install_yamls_out_dir }}, state=directory, mode=0755] *** 2026-01-21 11:08:30.901830 | controller | Wednesday 21 January 2026 11:08:30 +0000 (0:00:00.664) 0:01:14.315 ***** 2026-01-21 11:08:30.901839 | controller | Wednesday 21 January 2026 11:08:30 +0000 (0:00:00.664) 0:01:14.314 ***** 2026-01-21 11:08:30.901860 | controller | ok: [controller] 2026-01-21 11:08:30.960872 | controller | 2026-01-21 11:08:30.960949 | controller | TASK [install_yamls : Ensure user cifmw_install_yamls_vars contains existing Makefile variables that=_cifmw_install_yamls_unmatched_vars | length == 0, msg=cifmw_install_yamls_vars contains a variable that is not defined in install_yamls Makefile nor cifmw_install_yamls_whitelisted_vars: {{ _cifmw_install_yamls_unmatched_vars | join(', ')}}, quiet=True] *** 2026-01-21 11:08:30.960961 | controller | Wednesday 21 January 2026 11:08:30 +0000 (0:00:00.257) 0:01:14.573 ***** 2026-01-21 11:08:30.960971 | controller | Wednesday 21 January 2026 11:08:30 +0000 (0:00:00.257) 0:01:14.571 ***** 2026-01-21 11:08:30.960993 | controller | ok: [controller] 2026-01-21 11:08:31.719308 | controller | 2026-01-21 11:08:31.719411 | controller | TASK [install_yamls : Generate /home/zuul/ci-framework-data/artifacts/install_yamls.sh dest={{ cifmw_install_yamls_out_dir }}/{{ cifmw_install_yamls_envfile }}, content={% for k,v in cifmw_install_yamls_environment.items() %} 2026-01-21 11:08:31.719423 | controller | export {{ k }}={{ v }} 2026-01-21 11:08:31.719433 | controller | {% endfor %}, mode=0644] *** 2026-01-21 11:08:31.719457 | controller | Wednesday 21 January 2026 11:08:30 +0000 (0:00:00.059) 0:01:14.632 ***** 2026-01-21 11:08:31.719467 | controller | Wednesday 21 January 2026 11:08:30 +0000 (0:00:00.059) 0:01:14.631 ***** 2026-01-21 11:08:31.719502 | controller | changed: [controller] 2026-01-21 11:08:31.762077 | controller | 2026-01-21 11:08:31.762119 | controller | TASK [install_yamls : Set install_yamls default values cifmw_install_yamls_defaults={{ get_makefiles_env_output.makefiles_values | combine(cifmw_install_yamls_environment) }}, cacheable=True] *** 2026-01-21 11:08:31.762137 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.758) 0:01:15.390 ***** 2026-01-21 11:08:31.762145 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.758) 0:01:15.389 ***** 2026-01-21 11:08:31.762156 | controller | ok: [controller] 2026-01-21 11:08:31.791373 | controller | 2026-01-21 11:08:31.791419 | controller | TASK [install_yamls : Show the env structure var=cifmw_install_yamls_environment] *** 2026-01-21 11:08:31.791430 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.043) 0:01:15.433 ***** 2026-01-21 11:08:31.791439 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.043) 0:01:15.432 ***** 2026-01-21 11:08:31.791455 | controller | ok: [controller] => 2026-01-21 11:08:31.823599 | controller | cifmw_install_yamls_environment: 2026-01-21 11:08:31.823662 | controller | BMO_SETUP: false 2026-01-21 11:08:31.823672 | controller | CHECKOUT_FROM_OPENSTACK_REF: 'true' 2026-01-21 11:08:31.823681 | controller | OPENSTACK_K8S_BRANCH: main 2026-01-21 11:08:31.823689 | controller | OUT: /home/zuul/ci-framework-data/artifacts/manifests 2026-01-21 11:08:31.823733 | controller | OUTPUT_DIR: /home/zuul/ci-framework-data/artifacts/edpm 2026-01-21 11:08:31.823747 | controller | WATCHER_BRANCH: '' 2026-01-21 11:08:31.823756 | controller | WATCHER_REPO: /home/zuul/src/github.com/openstack-k8s-operators/watcher-operator 2026-01-21 11:08:31.823764 | controller | 2026-01-21 11:08:31.823773 | controller | TASK [install_yamls : Show the env structure defaults var=cifmw_install_yamls_defaults] *** 2026-01-21 11:08:31.823781 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.029) 0:01:15.462 ***** 2026-01-21 11:08:31.823790 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.029) 0:01:15.461 ***** 2026-01-21 11:08:31.823818 | controller | ok: [controller] => 2026-01-21 11:08:31.823826 | controller | cifmw_install_yamls_defaults: 2026-01-21 11:08:31.823835 | controller | ADOPTED_EXTERNAL_NETWORK: 172.21.1.0/24 2026-01-21 11:08:31.823843 | controller | ADOPTED_INTERNALAPI_NETWORK: 172.17.1.0/24 2026-01-21 11:08:31.823852 | controller | ADOPTED_STORAGEMGMT_NETWORK: 172.20.1.0/24 2026-01-21 11:08:31.823860 | controller | ADOPTED_STORAGE_NETWORK: 172.18.1.0/24 2026-01-21 11:08:31.823869 | controller | ADOPTED_TENANT_NETWORK: 172.9.1.0/24 2026-01-21 11:08:31.823879 | controller | ANSIBLEEE: config/samples/_v1beta1_ansibleee.yaml 2026-01-21 11:08:31.823887 | controller | ANSIBLEEE_BRANCH: main 2026-01-21 11:08:31.823895 | controller | ANSIBLEEE_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/openstack-ansibleee-operator/config/samples/_v1beta1_ansibleee.yaml 2026-01-21 11:08:31.823904 | controller | ANSIBLEEE_IMG: quay.io/openstack-k8s-operators/openstack-ansibleee-operator-index:latest 2026-01-21 11:08:31.823912 | controller | ANSIBLEEE_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/openstack-ansibleee-operator/kuttl-test.yaml 2026-01-21 11:08:31.823921 | controller | ANSIBLEEE_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/openstack-ansibleee-operator/test/kuttl/tests 2026-01-21 11:08:31.823929 | controller | ANSIBLEEE_KUTTL_NAMESPACE: ansibleee-kuttl-tests 2026-01-21 11:08:31.823937 | controller | ANSIBLEEE_REPO: https://github.com/openstack-k8s-operators/openstack-ansibleee-operator 2026-01-21 11:08:31.823946 | controller | ANSIBLEE_COMMIT_HASH: '' 2026-01-21 11:08:31.823954 | controller | BARBICAN: config/samples/barbican_v1beta1_barbican.yaml 2026-01-21 11:08:31.823961 | controller | BARBICAN_BRANCH: main 2026-01-21 11:08:31.823970 | controller | BARBICAN_COMMIT_HASH: '' 2026-01-21 11:08:31.823977 | controller | BARBICAN_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/barbican-operator/config/samples/barbican_v1beta1_barbican.yaml 2026-01-21 11:08:31.823985 | controller | BARBICAN_DEPL_IMG: unused 2026-01-21 11:08:31.823993 | controller | BARBICAN_IMG: quay.io/openstack-k8s-operators/barbican-operator-index:latest 2026-01-21 11:08:31.824001 | controller | BARBICAN_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/barbican-operator/kuttl-test.yaml 2026-01-21 11:08:31.824010 | controller | BARBICAN_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/barbican-operator/test/kuttl/tests 2026-01-21 11:08:31.824018 | controller | BARBICAN_KUTTL_NAMESPACE: barbican-kuttl-tests 2026-01-21 11:08:31.824038 | controller | BARBICAN_REPO: https://github.com/openstack-k8s-operators/barbican-operator.git 2026-01-21 11:08:31.824046 | controller | BARBICAN_SERVICE_ENABLED: 'true' 2026-01-21 11:08:31.824054 | controller | BARBICAN_SIMPLE_CRYPTO_ENCRYPTION_KEY: sEFmdFjDUqRM2VemYslV5yGNWjokioJXsg8Nrlc3drU= 2026-01-21 11:08:31.824061 | controller | BAREMETAL_BRANCH: main 2026-01-21 11:08:31.824069 | controller | BAREMETAL_COMMIT_HASH: '' 2026-01-21 11:08:31.824077 | controller | BAREMETAL_IMG: quay.io/openstack-k8s-operators/openstack-baremetal-operator-index:latest 2026-01-21 11:08:31.824085 | controller | BAREMETAL_OS_CONTAINER_IMG: '' 2026-01-21 11:08:31.824094 | controller | BAREMETAL_OS_IMG: '' 2026-01-21 11:08:31.824102 | controller | BAREMETAL_OS_IMG_TYPE: '' 2026-01-21 11:08:31.824110 | controller | BAREMETAL_REPO: https://github.com/openstack-k8s-operators/openstack-baremetal-operator.git 2026-01-21 11:08:31.824118 | controller | BAREMETAL_TIMEOUT: 20m 2026-01-21 11:08:31.824126 | controller | BASH_IMG: quay.io/openstack-k8s-operators/bash:latest 2026-01-21 11:08:31.824134 | controller | BGP_ASN: '64999' 2026-01-21 11:08:31.824142 | controller | BGP_LEAF_1: 100.65.4.1 2026-01-21 11:08:31.824150 | controller | BGP_LEAF_2: 100.64.4.1 2026-01-21 11:08:31.824158 | controller | BGP_OVN_ROUTING: 'false' 2026-01-21 11:08:31.824168 | controller | BGP_PEER_ASN: '64999' 2026-01-21 11:08:31.824176 | controller | BGP_SOURCE_IP: 172.30.4.2 2026-01-21 11:08:31.824184 | controller | BGP_SOURCE_IP6: f00d:f00d:f00d:f00d:f00d:f00d:f00d:42 2026-01-21 11:08:31.824192 | controller | BMAAS_BRIDGE_IPV4_PREFIX: 172.20.1.2/24 2026-01-21 11:08:31.824200 | controller | BMAAS_BRIDGE_IPV6_PREFIX: fd00:bbbb::2/64 2026-01-21 11:08:31.824208 | controller | BMAAS_INSTANCE_DISK_SIZE: '20' 2026-01-21 11:08:31.824216 | controller | BMAAS_INSTANCE_MEMORY: '4096' 2026-01-21 11:08:31.824224 | controller | BMAAS_INSTANCE_NAME_PREFIX: crc-bmaas 2026-01-21 11:08:31.824232 | controller | BMAAS_INSTANCE_NET_MODEL: virtio 2026-01-21 11:08:31.824240 | controller | BMAAS_INSTANCE_OS_VARIANT: centos-stream9 2026-01-21 11:08:31.824248 | controller | BMAAS_INSTANCE_VCPUS: '2' 2026-01-21 11:08:31.824256 | controller | BMAAS_INSTANCE_VIRT_TYPE: kvm 2026-01-21 11:08:31.824264 | controller | BMAAS_IPV4: 'true' 2026-01-21 11:08:31.824272 | controller | BMAAS_IPV6: 'false' 2026-01-21 11:08:31.824280 | controller | BMAAS_LIBVIRT_USER: sushyemu 2026-01-21 11:08:31.824288 | controller | BMAAS_METALLB_ADDRESS_POOL: 172.20.1.64/26 2026-01-21 11:08:31.824296 | controller | BMAAS_METALLB_POOL_NAME: baremetal 2026-01-21 11:08:31.824304 | controller | BMAAS_NETWORK_IPV4_PREFIX: 172.20.1.1/24 2026-01-21 11:08:31.824312 | controller | BMAAS_NETWORK_IPV6_PREFIX: fd00:bbbb::1/64 2026-01-21 11:08:31.824320 | controller | BMAAS_NETWORK_NAME: crc-bmaas 2026-01-21 11:08:31.824328 | controller | BMAAS_NODE_COUNT: '1' 2026-01-21 11:08:31.824336 | controller | BMAAS_OCP_INSTANCE_NAME: crc 2026-01-21 11:08:31.824344 | controller | BMAAS_REDFISH_PASSWORD: password 2026-01-21 11:08:31.824352 | controller | BMAAS_REDFISH_USERNAME: admin 2026-01-21 11:08:31.824360 | controller | BMAAS_ROUTE_LIBVIRT_NETWORKS: crc-bmaas,crc,default 2026-01-21 11:08:31.824368 | controller | BMAAS_SUSHY_EMULATOR_DRIVER: libvirt 2026-01-21 11:08:31.824376 | controller | BMAAS_SUSHY_EMULATOR_IMAGE: quay.io/metal3-io/sushy-tools:latest 2026-01-21 11:08:31.824384 | controller | BMAAS_SUSHY_EMULATOR_NAMESPACE: sushy-emulator 2026-01-21 11:08:31.824392 | controller | BMAAS_SUSHY_EMULATOR_OS_CLIENT_CONFIG_FILE: /etc/openstack/clouds.yaml 2026-01-21 11:08:31.824400 | controller | BMAAS_SUSHY_EMULATOR_OS_CLOUD: openstack 2026-01-21 11:08:31.824408 | controller | BMH_NAMESPACE: openstack 2026-01-21 11:08:31.824418 | controller | BMO_BRANCH: release-0.9 2026-01-21 11:08:31.824426 | controller | BMO_CLEANUP: 'true' 2026-01-21 11:08:31.824434 | controller | BMO_COMMIT_HASH: '' 2026-01-21 11:08:31.824442 | controller | BMO_IPA_BRANCH: stable/2024.1 2026-01-21 11:08:31.824449 | controller | BMO_IRONIC_HOST: 192.168.122.10 2026-01-21 11:08:31.824457 | controller | BMO_PROVISIONING_INTERFACE: '' 2026-01-21 11:08:31.824465 | controller | BMO_REPO: https://github.com/metal3-io/baremetal-operator 2026-01-21 11:08:31.824473 | controller | BMO_SETUP: false 2026-01-21 11:08:31.824487 | controller | BMO_SETUP_ROUTE_REPLACE: 'true' 2026-01-21 11:08:31.824496 | controller | BM_CTLPLANE_INTERFACE: enp1s0 2026-01-21 11:08:31.824511 | controller | BM_INSTANCE_MEMORY: '8192' 2026-01-21 11:08:31.824520 | controller | BM_INSTANCE_NAME_PREFIX: edpm-compute-baremetal 2026-01-21 11:08:31.824528 | controller | BM_INSTANCE_NAME_SUFFIX: '0' 2026-01-21 11:08:31.824536 | controller | BM_NETWORK_NAME: default 2026-01-21 11:08:31.824544 | controller | BM_NODE_COUNT: '1' 2026-01-21 11:08:31.824552 | controller | BM_ROOT_PASSWORD: '' 2026-01-21 11:08:31.824560 | controller | BM_ROOT_PASSWORD_SECRET: '' 2026-01-21 11:08:31.824568 | controller | CEILOMETER_CENTRAL_DEPL_IMG: unused 2026-01-21 11:08:31.824576 | controller | CEILOMETER_NOTIFICATION_DEPL_IMG: unused 2026-01-21 11:08:31.824613 | controller | CEPH_BRANCH: release-1.15 2026-01-21 11:08:31.824623 | controller | CEPH_CLIENT: /home/zuul/ci-framework-data/artifacts/manifests/operator/rook/deploy/examples/toolbox.yaml 2026-01-21 11:08:31.824631 | controller | CEPH_COMMON: /home/zuul/ci-framework-data/artifacts/manifests/operator/rook/deploy/examples/common.yaml 2026-01-21 11:08:31.824639 | controller | CEPH_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/rook/deploy/examples/cluster-test.yaml 2026-01-21 11:08:31.824647 | controller | CEPH_CRDS: /home/zuul/ci-framework-data/artifacts/manifests/operator/rook/deploy/examples/crds.yaml 2026-01-21 11:08:31.824655 | controller | CEPH_IMG: quay.io/ceph/demo:latest-squid 2026-01-21 11:08:31.824664 | controller | CEPH_OP: /home/zuul/ci-framework-data/artifacts/manifests/operator/rook/deploy/examples/operator-openshift.yaml 2026-01-21 11:08:31.824672 | controller | CEPH_REPO: https://github.com/rook/rook.git 2026-01-21 11:08:31.824680 | controller | CERTMANAGER_TIMEOUT: 300s 2026-01-21 11:08:31.824688 | controller | CHECKOUT_FROM_OPENSTACK_REF: 'true' 2026-01-21 11:08:31.824695 | controller | CINDER: config/samples/cinder_v1beta1_cinder.yaml 2026-01-21 11:08:31.824703 | controller | CINDERAPI_DEPL_IMG: unused 2026-01-21 11:08:31.824731 | controller | CINDERBKP_DEPL_IMG: unused 2026-01-21 11:08:31.824747 | controller | CINDERSCH_DEPL_IMG: unused 2026-01-21 11:08:31.824756 | controller | CINDERVOL_DEPL_IMG: unused 2026-01-21 11:08:31.824765 | controller | CINDER_BRANCH: main 2026-01-21 11:08:31.824773 | controller | CINDER_COMMIT_HASH: '' 2026-01-21 11:08:31.824781 | controller | CINDER_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/cinder-operator/config/samples/cinder_v1beta1_cinder.yaml 2026-01-21 11:08:31.824789 | controller | CINDER_IMG: quay.io/openstack-k8s-operators/cinder-operator-index:latest 2026-01-21 11:08:31.824797 | controller | CINDER_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/cinder-operator/kuttl-test.yaml 2026-01-21 11:08:31.824805 | controller | CINDER_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/cinder-operator/test/kuttl/tests 2026-01-21 11:08:31.824813 | controller | CINDER_KUTTL_NAMESPACE: cinder-kuttl-tests 2026-01-21 11:08:31.824822 | controller | CINDER_REPO: https://github.com/openstack-k8s-operators/cinder-operator.git 2026-01-21 11:08:31.824830 | controller | CLEANUP_DIR_CMD: rm -Rf 2026-01-21 11:08:31.824839 | controller | CRC_BGP_NIC_1_MAC: '52:54:00:11:11:11' 2026-01-21 11:08:31.824847 | controller | CRC_BGP_NIC_2_MAC: '52:54:00:11:11:12' 2026-01-21 11:08:31.824861 | controller | CRC_HTTPS_PROXY: '' 2026-01-21 11:08:31.824870 | controller | CRC_HTTP_PROXY: '' 2026-01-21 11:08:31.824878 | controller | CRC_STORAGE_NAMESPACE: crc-storage 2026-01-21 11:08:31.824885 | controller | CRC_STORAGE_RETRIES: '3' 2026-01-21 11:08:31.824894 | controller | CRC_URL: '''https://developers.redhat.com/content-gateway/rest/mirror/pub/openshift-v4/clients/crc/latest/crc-linux-amd64.tar.xz''' 2026-01-21 11:08:31.824902 | controller | CRC_VERSION: latest 2026-01-21 11:08:31.824914 | controller | DATAPLANE_ANSIBLE_SECRET: dataplane-ansible-ssh-private-key-secret 2026-01-21 11:08:31.824923 | controller | DATAPLANE_ANSIBLE_USER: '' 2026-01-21 11:08:31.824932 | controller | DATAPLANE_COMPUTE_IP: 192.168.122.100 2026-01-21 11:08:31.824940 | controller | DATAPLANE_CONTAINER_PREFIX: openstack 2026-01-21 11:08:31.824949 | controller | DATAPLANE_CONTAINER_TAG: current-podified 2026-01-21 11:08:31.824957 | controller | DATAPLANE_CUSTOM_SERVICE_RUNNER_IMG: quay.io/openstack-k8s-operators/openstack-ansibleee-runner:latest 2026-01-21 11:08:31.824966 | controller | DATAPLANE_DEFAULT_GW: 192.168.122.1 2026-01-21 11:08:31.824981 | controller | DATAPLANE_EXTRA_NOVA_CONFIG_FILE: /dev/null 2026-01-21 11:08:31.824990 | controller | DATAPLANE_GROWVOLS_ARGS: /=8GB /tmp=1GB /home=1GB /var=100% 2026-01-21 11:08:31.824999 | controller | DATAPLANE_KUSTOMIZE_SCENARIO: preprovisioned 2026-01-21 11:08:31.825007 | controller | DATAPLANE_NETWORKER_IP: 192.168.122.200 2026-01-21 11:08:31.825016 | controller | DATAPLANE_NETWORK_INTERFACE_NAME: eth0 2026-01-21 11:08:31.825025 | controller | DATAPLANE_NOVA_NFS_PATH: '' 2026-01-21 11:08:31.825034 | controller | DATAPLANE_NTP_SERVER: pool.ntp.org 2026-01-21 11:08:31.825043 | controller | DATAPLANE_PLAYBOOK: osp.edpm.download_cache 2026-01-21 11:08:31.825051 | controller | DATAPLANE_REGISTRY_URL: quay.io/podified-antelope-centos9 2026-01-21 11:08:31.825059 | controller | DATAPLANE_RUNNER_IMG: '' 2026-01-21 11:08:31.825067 | controller | DATAPLANE_SERVER_ROLE: compute 2026-01-21 11:08:31.825075 | controller | DATAPLANE_SSHD_ALLOWED_RANGES: '[''192.168.122.0/24'']' 2026-01-21 11:08:31.825083 | controller | DATAPLANE_TIMEOUT: 30m 2026-01-21 11:08:31.825092 | controller | DATAPLANE_TLS_ENABLED: 'true' 2026-01-21 11:08:31.825100 | controller | DATAPLANE_TOTAL_NETWORKER_NODES: '1' 2026-01-21 11:08:31.825108 | controller | DATAPLANE_TOTAL_NODES: '1' 2026-01-21 11:08:31.825116 | controller | DBSERVICE: galera 2026-01-21 11:08:31.825124 | controller | DESIGNATE: config/samples/designate_v1beta1_designate.yaml 2026-01-21 11:08:31.825132 | controller | DESIGNATE_BRANCH: main 2026-01-21 11:08:31.825140 | controller | DESIGNATE_COMMIT_HASH: '' 2026-01-21 11:08:31.825148 | controller | DESIGNATE_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/designate-operator/config/samples/designate_v1beta1_designate.yaml 2026-01-21 11:08:31.825171 | controller | DESIGNATE_IMG: quay.io/openstack-k8s-operators/designate-operator-index:latest 2026-01-21 11:08:31.825180 | controller | DESIGNATE_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/designate-operator/kuttl-test.yaml 2026-01-21 11:08:31.825197 | controller | DESIGNATE_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/designate-operator/test/kuttl/tests 2026-01-21 11:08:31.825205 | controller | DESIGNATE_KUTTL_NAMESPACE: designate-kuttl-tests 2026-01-21 11:08:31.825211 | controller | DESIGNATE_REPO: https://github.com/openstack-k8s-operators/designate-operator.git 2026-01-21 11:08:31.825217 | controller | DNSDATA: config/samples/network_v1beta1_dnsdata.yaml 2026-01-21 11:08:31.825224 | controller | DNSDATA_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator/config/samples/network_v1beta1_dnsdata.yaml 2026-01-21 11:08:31.825230 | controller | DNSMASQ: config/samples/network_v1beta1_dnsmasq.yaml 2026-01-21 11:08:31.825238 | controller | DNSMASQ_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator/config/samples/network_v1beta1_dnsmasq.yaml 2026-01-21 11:08:31.825244 | controller | DNS_DEPL_IMG: unused 2026-01-21 11:08:31.825251 | controller | DNS_DOMAIN: localdomain 2026-01-21 11:08:31.825257 | controller | DOWNLOAD_TOOLS_SELECTION: all 2026-01-21 11:08:31.825263 | controller | EDPM_ATTACH_EXTNET: 'true' 2026-01-21 11:08:31.825269 | controller | EDPM_COMPUTE_ADDITIONAL_HOST_ROUTES: '''[]''' 2026-01-21 11:08:31.825276 | controller | EDPM_COMPUTE_ADDITIONAL_NETWORKS: '''[]''' 2026-01-21 11:08:31.825282 | controller | EDPM_COMPUTE_CELLS: '1' 2026-01-21 11:08:31.825288 | controller | EDPM_COMPUTE_CEPH_ENABLED: 'true' 2026-01-21 11:08:31.825294 | controller | EDPM_COMPUTE_CEPH_NOVA: 'true' 2026-01-21 11:08:31.825301 | controller | EDPM_COMPUTE_DHCP_AGENT_ENABLED: 'true' 2026-01-21 11:08:31.825307 | controller | EDPM_COMPUTE_SRIOV_ENABLED: 'true' 2026-01-21 11:08:31.825313 | controller | EDPM_COMPUTE_SUFFIX: '0' 2026-01-21 11:08:31.825319 | controller | EDPM_CONFIGURE_DEFAULT_ROUTE: 'true' 2026-01-21 11:08:31.825326 | controller | EDPM_CONFIGURE_HUGEPAGES: 'false' 2026-01-21 11:08:31.825332 | controller | EDPM_CONFIGURE_NETWORKING: 'true' 2026-01-21 11:08:31.825338 | controller | EDPM_FIRSTBOOT_EXTRA: /tmp/edpm-firstboot-extra 2026-01-21 11:08:31.825344 | controller | EDPM_NETWORKER_SUFFIX: '0' 2026-01-21 11:08:31.825350 | controller | EDPM_TOTAL_NETWORKERS: '1' 2026-01-21 11:08:31.825357 | controller | EDPM_TOTAL_NODES: '1' 2026-01-21 11:08:31.825363 | controller | GALERA_REPLICAS: '' 2026-01-21 11:08:31.825369 | controller | GENERATE_SSH_KEYS: 'true' 2026-01-21 11:08:31.825380 | controller | GIT_CLONE_OPTS: '' 2026-01-21 11:08:31.825387 | controller | GLANCE: config/samples/glance_v1beta1_glance.yaml 2026-01-21 11:08:31.825393 | controller | GLANCEAPI_DEPL_IMG: unused 2026-01-21 11:08:31.825400 | controller | GLANCE_BRANCH: main 2026-01-21 11:08:31.825406 | controller | GLANCE_COMMIT_HASH: '' 2026-01-21 11:08:31.825412 | controller | GLANCE_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/glance-operator/config/samples/glance_v1beta1_glance.yaml 2026-01-21 11:08:31.825419 | controller | GLANCE_IMG: quay.io/openstack-k8s-operators/glance-operator-index:latest 2026-01-21 11:08:31.825425 | controller | GLANCE_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/glance-operator/kuttl-test.yaml 2026-01-21 11:08:31.825432 | controller | GLANCE_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/glance-operator/test/kuttl/tests 2026-01-21 11:08:31.825438 | controller | GLANCE_KUTTL_NAMESPACE: glance-kuttl-tests 2026-01-21 11:08:31.825445 | controller | GLANCE_REPO: https://github.com/openstack-k8s-operators/glance-operator.git 2026-01-21 11:08:31.825451 | controller | HEAT: config/samples/heat_v1beta1_heat.yaml 2026-01-21 11:08:31.825461 | controller | HEATAPI_DEPL_IMG: unused 2026-01-21 11:08:31.825468 | controller | HEATCFNAPI_DEPL_IMG: unused 2026-01-21 11:08:31.825474 | controller | HEATENGINE_DEPL_IMG: unused 2026-01-21 11:08:31.825480 | controller | HEAT_AUTH_ENCRYPTION_KEY: 767c3ed056cbaa3b9dfedb8c6f825bf0 2026-01-21 11:08:31.825488 | controller | HEAT_BRANCH: main 2026-01-21 11:08:31.825495 | controller | HEAT_COMMIT_HASH: '' 2026-01-21 11:08:31.825502 | controller | HEAT_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/heat-operator/config/samples/heat_v1beta1_heat.yaml 2026-01-21 11:08:31.825510 | controller | HEAT_IMG: quay.io/openstack-k8s-operators/heat-operator-index:latest 2026-01-21 11:08:31.825517 | controller | HEAT_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/heat-operator/kuttl-test.yaml 2026-01-21 11:08:31.825523 | controller | HEAT_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/heat-operator/test/kuttl/tests 2026-01-21 11:08:31.825530 | controller | HEAT_KUTTL_NAMESPACE: heat-kuttl-tests 2026-01-21 11:08:31.825536 | controller | HEAT_REPO: https://github.com/openstack-k8s-operators/heat-operator.git 2026-01-21 11:08:31.825543 | controller | HEAT_SERVICE_ENABLED: 'true' 2026-01-21 11:08:31.825549 | controller | HORIZON: config/samples/horizon_v1beta1_horizon.yaml 2026-01-21 11:08:31.825555 | controller | HORIZON_BRANCH: main 2026-01-21 11:08:31.825562 | controller | HORIZON_COMMIT_HASH: '' 2026-01-21 11:08:31.825568 | controller | HORIZON_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/horizon-operator/config/samples/horizon_v1beta1_horizon.yaml 2026-01-21 11:08:31.825574 | controller | HORIZON_DEPL_IMG: unused 2026-01-21 11:08:31.825580 | controller | HORIZON_IMG: quay.io/openstack-k8s-operators/horizon-operator-index:latest 2026-01-21 11:08:31.825587 | controller | HORIZON_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/horizon-operator/kuttl-test.yaml 2026-01-21 11:08:31.825601 | controller | HORIZON_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/horizon-operator/test/kuttl/tests 2026-01-21 11:08:31.825607 | controller | HORIZON_KUTTL_NAMESPACE: horizon-kuttl-tests 2026-01-21 11:08:31.825614 | controller | HORIZON_REPO: https://github.com/openstack-k8s-operators/horizon-operator.git 2026-01-21 11:08:31.825620 | controller | INFRA_BRANCH: main 2026-01-21 11:08:31.825626 | controller | INFRA_COMMIT_HASH: '' 2026-01-21 11:08:31.825633 | controller | INFRA_IMG: quay.io/openstack-k8s-operators/infra-operator-index:latest 2026-01-21 11:08:31.825639 | controller | INFRA_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator/kuttl-test.yaml 2026-01-21 11:08:31.825645 | controller | INFRA_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator/test/kuttl/tests 2026-01-21 11:08:31.825652 | controller | INFRA_KUTTL_NAMESPACE: infra-kuttl-tests 2026-01-21 11:08:31.825662 | controller | INFRA_REPO: https://github.com/openstack-k8s-operators/infra-operator.git 2026-01-21 11:08:31.825668 | controller | INSTALL_CERT_MANAGER: 'true' 2026-01-21 11:08:31.825685 | controller | INSTALL_NMSTATE: true || false 2026-01-21 11:08:31.825692 | controller | INSTALL_NNCP: true || false 2026-01-21 11:08:31.825698 | controller | INTERNALAPI_HOST_ROUTES: '' 2026-01-21 11:08:31.825705 | controller | IPV6_LAB_IPV4_NETWORK_IPADDRESS: 172.30.0.1/24 2026-01-21 11:08:31.825730 | controller | IPV6_LAB_IPV6_NETWORK_IPADDRESS: fd00:abcd:abcd:fc00::1/64 2026-01-21 11:08:31.825742 | controller | IPV6_LAB_LIBVIRT_STORAGE_POOL: default 2026-01-21 11:08:31.825749 | controller | IPV6_LAB_MANAGE_FIREWALLD: 'true' 2026-01-21 11:08:31.825755 | controller | IPV6_LAB_NAT64_HOST_IPV4: 172.30.0.2/24 2026-01-21 11:08:31.825761 | controller | IPV6_LAB_NAT64_HOST_IPV6: fd00:abcd:abcd:fc00::2/64 2026-01-21 11:08:31.825768 | controller | IPV6_LAB_NAT64_INSTANCE_NAME: nat64-router 2026-01-21 11:08:31.825783 | controller | IPV6_LAB_NAT64_IPV6_NETWORK: fd00:abcd:abcd:fc00::/64 2026-01-21 11:08:31.825789 | controller | IPV6_LAB_NAT64_TAYGA_DYNAMIC_POOL: 192.168.255.0/24 2026-01-21 11:08:31.825796 | controller | IPV6_LAB_NAT64_TAYGA_IPV4: 192.168.255.1 2026-01-21 11:08:31.825802 | controller | IPV6_LAB_NAT64_TAYGA_IPV6: fd00:abcd:abcd:fc00::3 2026-01-21 11:08:31.825808 | controller | IPV6_LAB_NAT64_TAYGA_IPV6_PREFIX: fd00:abcd:abcd:fcff::/96 2026-01-21 11:08:31.825815 | controller | IPV6_LAB_NAT64_UPDATE_PACKAGES: 'false' 2026-01-21 11:08:31.825821 | controller | IPV6_LAB_NETWORK_NAME: nat64 2026-01-21 11:08:31.825827 | controller | IPV6_LAB_SNO_CLUSTER_NETWORK: fd00:abcd:0::/48 2026-01-21 11:08:31.825833 | controller | IPV6_LAB_SNO_HOST_IP: fd00:abcd:abcd:fc00::11 2026-01-21 11:08:31.825840 | controller | IPV6_LAB_SNO_HOST_PREFIX: '64' 2026-01-21 11:08:31.825846 | controller | IPV6_LAB_SNO_INSTANCE_NAME: sno 2026-01-21 11:08:31.825852 | controller | IPV6_LAB_SNO_MACHINE_NETWORK: fd00:abcd:abcd:fc00::/64 2026-01-21 11:08:31.825859 | controller | IPV6_LAB_SNO_OCP_MIRROR_URL: https://mirror.openshift.com/pub/openshift-v4/clients/ocp 2026-01-21 11:08:31.825865 | controller | IPV6_LAB_SNO_OCP_VERSION: latest-4.14 2026-01-21 11:08:31.825871 | controller | IPV6_LAB_SNO_SERVICE_NETWORK: fd00:abcd:abcd:fc03::/112 2026-01-21 11:08:31.825877 | controller | IPV6_LAB_SSH_PUB_KEY: /home/zuul/.ssh/id_rsa.pub 2026-01-21 11:08:31.825884 | controller | IPV6_LAB_WORK_DIR: /home/zuul/.ipv6lab 2026-01-21 11:08:31.825890 | controller | IRONIC: config/samples/ironic_v1beta1_ironic.yaml 2026-01-21 11:08:31.825896 | controller | IRONICAPI_DEPL_IMG: unused 2026-01-21 11:08:31.825902 | controller | IRONICCON_DEPL_IMG: unused 2026-01-21 11:08:31.825909 | controller | IRONICINS_DEPL_IMG: unused 2026-01-21 11:08:31.825915 | controller | IRONICNAG_DEPL_IMG: unused 2026-01-21 11:08:31.825925 | controller | IRONICPXE_DEPL_IMG: unused 2026-01-21 11:08:31.825931 | controller | IRONIC_BRANCH: main 2026-01-21 11:08:31.825938 | controller | IRONIC_COMMIT_HASH: '' 2026-01-21 11:08:31.825944 | controller | IRONIC_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/ironic-operator/config/samples/ironic_v1beta1_ironic.yaml 2026-01-21 11:08:31.825951 | controller | IRONIC_IMAGE: quay.io/metal3-io/ironic 2026-01-21 11:08:31.825957 | controller | IRONIC_IMAGE_TAG: release-24.1 2026-01-21 11:08:31.825963 | controller | IRONIC_IMG: quay.io/openstack-k8s-operators/ironic-operator-index:latest 2026-01-21 11:08:31.825970 | controller | IRONIC_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/ironic-operator/kuttl-test.yaml 2026-01-21 11:08:31.825976 | controller | IRONIC_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/ironic-operator/test/kuttl/tests 2026-01-21 11:08:31.825982 | controller | IRONIC_KUTTL_NAMESPACE: ironic-kuttl-tests 2026-01-21 11:08:31.825989 | controller | IRONIC_REPO: https://github.com/openstack-k8s-operators/ironic-operator.git 2026-01-21 11:08:31.825995 | controller | KEYSTONEAPI: config/samples/keystone_v1beta1_keystoneapi.yaml 2026-01-21 11:08:31.826001 | controller | KEYSTONEAPI_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/keystone-operator/config/samples/keystone_v1beta1_keystoneapi.yaml 2026-01-21 11:08:31.826008 | controller | KEYSTONEAPI_DEPL_IMG: unused 2026-01-21 11:08:31.826014 | controller | KEYSTONE_BRANCH: main 2026-01-21 11:08:31.826021 | controller | KEYSTONE_COMMIT_HASH: '' 2026-01-21 11:08:31.826027 | controller | KEYSTONE_FEDERATION_CLIENT_SECRET: COX8bmlKAWn56XCGMrKQJj7dgHNAOl6f 2026-01-21 11:08:31.826037 | controller | KEYSTONE_FEDERATION_CRYPTO_PASSPHRASE: openstack 2026-01-21 11:08:31.826043 | controller | KEYSTONE_IMG: quay.io/openstack-k8s-operators/keystone-operator-index:latest 2026-01-21 11:08:31.826049 | controller | KEYSTONE_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/keystone-operator/kuttl-test.yaml 2026-01-21 11:08:31.826056 | controller | KEYSTONE_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/keystone-operator/test/kuttl/tests 2026-01-21 11:08:31.826062 | controller | KEYSTONE_KUTTL_NAMESPACE: keystone-kuttl-tests 2026-01-21 11:08:31.826068 | controller | KEYSTONE_REPO: https://github.com/openstack-k8s-operators/keystone-operator.git 2026-01-21 11:08:31.826075 | controller | KUBEADMIN_PWD: '12345678' 2026-01-21 11:08:31.826084 | controller | LIBVIRT_SECRET: libvirt-secret 2026-01-21 11:08:31.826092 | controller | LOKI_DEPLOY_MODE: openshift-network 2026-01-21 11:08:31.826100 | controller | LOKI_DEPLOY_NAMESPACE: netobserv 2026-01-21 11:08:31.826109 | controller | LOKI_DEPLOY_SIZE: 1x.demo 2026-01-21 11:08:31.826117 | controller | LOKI_NAMESPACE: openshift-operators-redhat 2026-01-21 11:08:31.826127 | controller | LOKI_OPERATOR_GROUP: openshift-operators-redhat-loki 2026-01-21 11:08:31.826135 | controller | LOKI_SUBSCRIPTION: loki-operator 2026-01-21 11:08:31.826143 | controller | LVMS_CR: '1' 2026-01-21 11:08:31.826149 | controller | MANILA: config/samples/manila_v1beta1_manila.yaml 2026-01-21 11:08:31.826155 | controller | MANILAAPI_DEPL_IMG: unused 2026-01-21 11:08:31.826163 | controller | MANILASCH_DEPL_IMG: unused 2026-01-21 11:08:31.826172 | controller | MANILASHARE_DEPL_IMG: unused 2026-01-21 11:08:31.826180 | controller | MANILA_BRANCH: main 2026-01-21 11:08:31.826188 | controller | MANILA_COMMIT_HASH: '' 2026-01-21 11:08:31.826196 | controller | MANILA_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/manila-operator/config/samples/manila_v1beta1_manila.yaml 2026-01-21 11:08:31.826205 | controller | MANILA_IMG: quay.io/openstack-k8s-operators/manila-operator-index:latest 2026-01-21 11:08:31.826217 | controller | MANILA_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/manila-operator/kuttl-test.yaml 2026-01-21 11:08:31.826224 | controller | MANILA_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/manila-operator/test/kuttl/tests 2026-01-21 11:08:31.826231 | controller | MANILA_KUTTL_NAMESPACE: manila-kuttl-tests 2026-01-21 11:08:31.826237 | controller | MANILA_REPO: https://github.com/openstack-k8s-operators/manila-operator.git 2026-01-21 11:08:31.826243 | controller | MANILA_SERVICE_ENABLED: 'true' 2026-01-21 11:08:31.826249 | controller | MARIADB: config/samples/mariadb_v1beta1_galera.yaml 2026-01-21 11:08:31.826256 | controller | MARIADB_BRANCH: main 2026-01-21 11:08:31.826266 | controller | MARIADB_CHAINSAW_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/mariadb-operator/test/chainsaw/config.yaml 2026-01-21 11:08:31.826273 | controller | MARIADB_CHAINSAW_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/mariadb-operator/test/chainsaw/tests 2026-01-21 11:08:31.826279 | controller | MARIADB_CHAINSAW_NAMESPACE: mariadb-chainsaw-tests 2026-01-21 11:08:31.826285 | controller | MARIADB_COMMIT_HASH: '' 2026-01-21 11:08:31.826292 | controller | MARIADB_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/mariadb-operator/config/samples/mariadb_v1beta1_galera.yaml 2026-01-21 11:08:31.826298 | controller | MARIADB_DEPL_IMG: unused 2026-01-21 11:08:31.826304 | controller | MARIADB_IMG: quay.io/openstack-k8s-operators/mariadb-operator-index:latest 2026-01-21 11:08:31.826311 | controller | MARIADB_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/mariadb-operator/kuttl-test.yaml 2026-01-21 11:08:31.826317 | controller | MARIADB_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/mariadb-operator/test/kuttl/tests 2026-01-21 11:08:31.826323 | controller | MARIADB_KUTTL_NAMESPACE: mariadb-kuttl-tests 2026-01-21 11:08:31.826330 | controller | MARIADB_REPO: https://github.com/openstack-k8s-operators/mariadb-operator.git 2026-01-21 11:08:31.826336 | controller | MEMCACHED: config/samples/memcached_v1beta1_memcached.yaml 2026-01-21 11:08:31.826342 | controller | MEMCACHED_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator/config/samples/memcached_v1beta1_memcached.yaml 2026-01-21 11:08:31.826352 | controller | MEMCACHED_DEPL_IMG: unused 2026-01-21 11:08:31.826358 | controller | METADATA_SHARED_SECRET: '1234567842' 2026-01-21 11:08:31.826366 | controller | METALLB_IPV6_POOL: fd00:aaaa::80-fd00:aaaa::90 2026-01-21 11:08:31.826372 | controller | METALLB_POOL: 192.168.122.80-192.168.122.90 2026-01-21 11:08:31.826379 | controller | MICROSHIFT: '0' 2026-01-21 11:08:31.826385 | controller | NAMESPACE: openstack 2026-01-21 11:08:31.826391 | controller | NETCONFIG: config/samples/network_v1beta1_netconfig.yaml 2026-01-21 11:08:31.826397 | controller | NETCONFIG_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator/config/samples/network_v1beta1_netconfig.yaml 2026-01-21 11:08:31.826403 | controller | NETCONFIG_DEPL_IMG: unused 2026-01-21 11:08:31.826410 | controller | NETOBSERV_DEPLOY_NAMESPACE: netobserv 2026-01-21 11:08:31.826416 | controller | NETOBSERV_NAMESPACE: openshift-netobserv-operator 2026-01-21 11:08:31.826422 | controller | NETOBSERV_OPERATOR_GROUP: openshift-netobserv-operator-net 2026-01-21 11:08:31.826429 | controller | NETOBSERV_SUBSCRIPTION: netobserv-operator 2026-01-21 11:08:31.826435 | controller | NETWORK_BGP: 'false' 2026-01-21 11:08:31.826441 | controller | NETWORK_DESIGNATE_ADDRESS_PREFIX: 172.28.0 2026-01-21 11:08:31.826448 | controller | NETWORK_DESIGNATE_EXT_ADDRESS_PREFIX: 172.50.0 2026-01-21 11:08:31.826454 | controller | NETWORK_INTERNALAPI_ADDRESS_PREFIX: 172.17.0 2026-01-21 11:08:31.826460 | controller | NETWORK_ISOLATION: 'true' 2026-01-21 11:08:31.826467 | controller | NETWORK_ISOLATION_INSTANCE_NAME: crc 2026-01-21 11:08:31.826473 | controller | NETWORK_ISOLATION_IPV4: 'true' 2026-01-21 11:08:31.826479 | controller | NETWORK_ISOLATION_IPV4_ADDRESS: 172.16.1.1/24 2026-01-21 11:08:31.826485 | controller | NETWORK_ISOLATION_IPV4_NAT: 'true' 2026-01-21 11:08:31.826492 | controller | NETWORK_ISOLATION_IPV6: 'false' 2026-01-21 11:08:31.826498 | controller | NETWORK_ISOLATION_IPV6_ADDRESS: fd00:aaaa::1/64 2026-01-21 11:08:31.826504 | controller | NETWORK_ISOLATION_IP_ADDRESS: 192.168.122.10 2026-01-21 11:08:31.826510 | controller | NETWORK_ISOLATION_MAC: '52:54:00:11:11:10' 2026-01-21 11:08:31.826516 | controller | NETWORK_ISOLATION_NETWORK_NAME: net-iso 2026-01-21 11:08:31.826523 | controller | NETWORK_ISOLATION_NET_NAME: default 2026-01-21 11:08:31.826529 | controller | NETWORK_ISOLATION_USE_DEFAULT_NETWORK: 'true' 2026-01-21 11:08:31.826535 | controller | NETWORK_MTU: '1500' 2026-01-21 11:08:31.826541 | controller | NETWORK_STORAGEMGMT_ADDRESS_PREFIX: 172.20.0 2026-01-21 11:08:31.826548 | controller | NETWORK_STORAGE_ADDRESS_PREFIX: 172.18.0 2026-01-21 11:08:31.826554 | controller | NETWORK_STORAGE_MACVLAN: '' 2026-01-21 11:08:31.826560 | controller | NETWORK_TENANT_ADDRESS_PREFIX: 172.19.0 2026-01-21 11:08:31.826567 | controller | NETWORK_VLAN_START: '20' 2026-01-21 11:08:31.826573 | controller | NETWORK_VLAN_STEP: '1' 2026-01-21 11:08:31.826579 | controller | NEUTRONAPI: config/samples/neutron_v1beta1_neutronapi.yaml 2026-01-21 11:08:31.826585 | controller | NEUTRONAPI_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/neutron-operator/config/samples/neutron_v1beta1_neutronapi.yaml 2026-01-21 11:08:31.826592 | controller | NEUTRONAPI_DEPL_IMG: unused 2026-01-21 11:08:31.826598 | controller | NEUTRON_BRANCH: main 2026-01-21 11:08:31.826604 | controller | NEUTRON_COMMIT_HASH: '' 2026-01-21 11:08:31.826610 | controller | NEUTRON_IMG: quay.io/openstack-k8s-operators/neutron-operator-index:latest 2026-01-21 11:08:31.826618 | controller | NEUTRON_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/neutron-operator/kuttl-test.yaml 2026-01-21 11:08:31.826626 | controller | NEUTRON_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/neutron-operator/test/kuttl/tests 2026-01-21 11:08:31.826634 | controller | NEUTRON_KUTTL_NAMESPACE: neutron-kuttl-tests 2026-01-21 11:08:31.826642 | controller | NEUTRON_REPO: https://github.com/openstack-k8s-operators/neutron-operator.git 2026-01-21 11:08:31.826660 | controller | NFS_HOME: /home/nfs 2026-01-21 11:08:31.826668 | controller | NMSTATE_NAMESPACE: openshift-nmstate 2026-01-21 11:08:31.826681 | controller | NMSTATE_OPERATOR_GROUP: openshift-nmstate-tn6k8 2026-01-21 11:08:31.826689 | controller | NMSTATE_SUBSCRIPTION: kubernetes-nmstate-operator 2026-01-21 11:08:31.826701 | controller | NNCP_ADDITIONAL_HOST_ROUTES: '' 2026-01-21 11:08:31.826729 | controller | NNCP_BGP_1_INTERFACE: enp7s0 2026-01-21 11:08:31.826743 | controller | NNCP_BGP_1_IP_ADDRESS: 100.65.4.2 2026-01-21 11:08:31.826751 | controller | NNCP_BGP_2_INTERFACE: enp8s0 2026-01-21 11:08:31.826759 | controller | NNCP_BGP_2_IP_ADDRESS: 100.64.4.2 2026-01-21 11:08:31.826767 | controller | NNCP_BRIDGE: ospbr 2026-01-21 11:08:31.826774 | controller | NNCP_CLEANUP_TIMEOUT: 120s 2026-01-21 11:08:31.826782 | controller | NNCP_CTLPLANE_IPV6_ADDRESS_PREFIX: 'fd00:aaaa::' 2026-01-21 11:08:31.826790 | controller | NNCP_CTLPLANE_IPV6_ADDRESS_SUFFIX: '10' 2026-01-21 11:08:31.826798 | controller | NNCP_CTLPLANE_IP_ADDRESS_PREFIX: 192.168.122 2026-01-21 11:08:31.826805 | controller | NNCP_CTLPLANE_IP_ADDRESS_SUFFIX: '10' 2026-01-21 11:08:31.826813 | controller | NNCP_DNS_SERVER: 192.168.122.1 2026-01-21 11:08:31.826821 | controller | NNCP_DNS_SERVER_IPV6: fd00:aaaa::1 2026-01-21 11:08:31.826829 | controller | NNCP_GATEWAY: 192.168.122.1 2026-01-21 11:08:31.826837 | controller | NNCP_GATEWAY_IPV6: fd00:aaaa::1 2026-01-21 11:08:31.826844 | controller | NNCP_INTERFACE: enp6s0 2026-01-21 11:08:31.826852 | controller | NNCP_NODES: '' 2026-01-21 11:08:31.826860 | controller | NNCP_TIMEOUT: 240s 2026-01-21 11:08:31.826868 | controller | NOVA: config/samples/nova_v1beta1_nova_collapsed_cell.yaml 2026-01-21 11:08:31.826875 | controller | NOVA_BRANCH: main 2026-01-21 11:08:31.826883 | controller | NOVA_COMMIT_HASH: '' 2026-01-21 11:08:31.826892 | controller | NOVA_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/nova-operator/config/samples/nova_v1beta1_nova_collapsed_cell.yaml 2026-01-21 11:08:31.826900 | controller | NOVA_IMG: quay.io/openstack-k8s-operators/nova-operator-index:latest 2026-01-21 11:08:31.826908 | controller | NOVA_REPO: https://github.com/openstack-k8s-operators/nova-operator.git 2026-01-21 11:08:31.826917 | controller | NUMBER_OF_INSTANCES: '1' 2026-01-21 11:08:31.826925 | controller | OCP_NETWORK_NAME: crc 2026-01-21 11:08:31.826931 | controller | OCTAVIA: config/samples/octavia_v1beta1_octavia.yaml 2026-01-21 11:08:31.826938 | controller | OCTAVIA_BRANCH: main 2026-01-21 11:08:31.826944 | controller | OCTAVIA_COMMIT_HASH: '' 2026-01-21 11:08:31.826964 | controller | OCTAVIA_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/octavia-operator/config/samples/octavia_v1beta1_octavia.yaml 2026-01-21 11:08:31.826971 | controller | OCTAVIA_IMG: quay.io/openstack-k8s-operators/octavia-operator-index:latest 2026-01-21 11:08:31.826977 | controller | OCTAVIA_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/octavia-operator/kuttl-test.yaml 2026-01-21 11:08:31.826984 | controller | OCTAVIA_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/octavia-operator/test/kuttl/tests 2026-01-21 11:08:31.826990 | controller | OCTAVIA_KUTTL_NAMESPACE: octavia-kuttl-tests 2026-01-21 11:08:31.826996 | controller | OCTAVIA_REPO: https://github.com/openstack-k8s-operators/octavia-operator.git 2026-01-21 11:08:31.827003 | controller | OKD: 'false' 2026-01-21 11:08:31.827009 | controller | OPENSTACK_BRANCH: main 2026-01-21 11:08:31.827015 | controller | OPENSTACK_BUNDLE_IMG: quay.io/openstack-k8s-operators/openstack-operator-bundle:latest 2026-01-21 11:08:31.827022 | controller | OPENSTACK_COMMIT_HASH: '' 2026-01-21 11:08:31.827028 | controller | OPENSTACK_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/openstack-operator/config/samples/core_v1beta1_openstackcontrolplane_galera_network_isolation.yaml 2026-01-21 11:08:31.827035 | controller | OPENSTACK_CRDS_DIR: openstack_crds 2026-01-21 11:08:31.827041 | controller | OPENSTACK_CTLPLANE: config/samples/core_v1beta1_openstackcontrolplane_galera_network_isolation.yaml 2026-01-21 11:08:31.827048 | controller | OPENSTACK_IMG: quay.io/openstack-k8s-operators/openstack-operator-index:latest 2026-01-21 11:08:31.827054 | controller | OPENSTACK_K8S_BRANCH: main 2026-01-21 11:08:31.827060 | controller | OPENSTACK_K8S_TAG: latest 2026-01-21 11:08:31.827067 | controller | OPENSTACK_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/openstack-operator/kuttl-test.yaml 2026-01-21 11:08:31.827073 | controller | OPENSTACK_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/openstack-operator/test/kuttl/tests 2026-01-21 11:08:31.827083 | controller | OPENSTACK_KUTTL_NAMESPACE: openstack-kuttl-tests 2026-01-21 11:08:31.827089 | controller | OPENSTACK_NEUTRON_CUSTOM_CONF: '' 2026-01-21 11:08:31.827096 | controller | OPENSTACK_REPO: https://github.com/openstack-k8s-operators/openstack-operator.git 2026-01-21 11:08:31.827104 | controller | OPENSTACK_STORAGE_BUNDLE_IMG: quay.io/openstack-k8s-operators/openstack-operator-storage-bundle:latest 2026-01-21 11:08:31.827110 | controller | OPERATOR_BASE_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator 2026-01-21 11:08:31.827116 | controller | OPERATOR_CHANNEL: '' 2026-01-21 11:08:31.827123 | controller | OPERATOR_NAMESPACE: openstack-operators 2026-01-21 11:08:31.827129 | controller | OPERATOR_SOURCE: '' 2026-01-21 11:08:31.827135 | controller | OPERATOR_SOURCE_NAMESPACE: '' 2026-01-21 11:08:31.827141 | controller | OUT: /home/zuul/ci-framework-data/artifacts/manifests 2026-01-21 11:08:31.827148 | controller | OUTPUT_DIR: /home/zuul/ci-framework-data/artifacts/edpm 2026-01-21 11:08:31.827154 | controller | OVNCONTROLLER: config/samples/ovn_v1beta1_ovncontroller.yaml 2026-01-21 11:08:31.827161 | controller | OVNCONTROLLER_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/ovn-operator/config/samples/ovn_v1beta1_ovncontroller.yaml 2026-01-21 11:08:31.827167 | controller | OVNCONTROLLER_NMAP: 'true' 2026-01-21 11:08:31.827187 | controller | OVNDBS: config/samples/ovn_v1beta1_ovndbcluster.yaml 2026-01-21 11:08:31.827196 | controller | OVNDBS_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/ovn-operator/config/samples/ovn_v1beta1_ovndbcluster.yaml 2026-01-21 11:08:31.827204 | controller | OVNNORTHD: config/samples/ovn_v1beta1_ovnnorthd.yaml 2026-01-21 11:08:31.827213 | controller | OVNNORTHD_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/ovn-operator/config/samples/ovn_v1beta1_ovnnorthd.yaml 2026-01-21 11:08:31.827222 | controller | OVN_BRANCH: main 2026-01-21 11:08:31.827237 | controller | OVN_COMMIT_HASH: '' 2026-01-21 11:08:31.827247 | controller | OVN_IMG: quay.io/openstack-k8s-operators/ovn-operator-index:latest 2026-01-21 11:08:31.827253 | controller | OVN_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/ovn-operator/kuttl-test.yaml 2026-01-21 11:08:31.827260 | controller | OVN_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/ovn-operator/test/kuttl/tests 2026-01-21 11:08:31.827270 | controller | OVN_KUTTL_NAMESPACE: ovn-kuttl-tests 2026-01-21 11:08:31.827277 | controller | OVN_REPO: https://github.com/openstack-k8s-operators/ovn-operator.git 2026-01-21 11:08:31.827283 | controller | PASSWORD: '12345678' 2026-01-21 11:08:31.827290 | controller | PLACEMENTAPI: config/samples/placement_v1beta1_placementapi.yaml 2026-01-21 11:08:31.827296 | controller | PLACEMENTAPI_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/placement-operator/config/samples/placement_v1beta1_placementapi.yaml 2026-01-21 11:08:31.827303 | controller | PLACEMENTAPI_DEPL_IMG: unused 2026-01-21 11:08:31.827309 | controller | PLACEMENT_BRANCH: main 2026-01-21 11:08:31.827315 | controller | PLACEMENT_COMMIT_HASH: '' 2026-01-21 11:08:31.827321 | controller | PLACEMENT_IMG: quay.io/openstack-k8s-operators/placement-operator-index:latest 2026-01-21 11:08:31.827328 | controller | PLACEMENT_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/placement-operator/kuttl-test.yaml 2026-01-21 11:08:31.827334 | controller | PLACEMENT_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/placement-operator/test/kuttl/tests 2026-01-21 11:08:31.827340 | controller | PLACEMENT_KUTTL_NAMESPACE: placement-kuttl-tests 2026-01-21 11:08:31.827346 | controller | PLACEMENT_REPO: https://github.com/openstack-k8s-operators/placement-operator.git 2026-01-21 11:08:31.827353 | controller | PULL_SECRET: /home/zuul/pull-secret.txt 2026-01-21 11:08:31.827359 | controller | RABBITMQ: docs/examples/default-security-context/rabbitmq.yaml 2026-01-21 11:08:31.827365 | controller | RABBITMQ_BRANCH: patches 2026-01-21 11:08:31.827371 | controller | RABBITMQ_COMMIT_HASH: '' 2026-01-21 11:08:31.827378 | controller | RABBITMQ_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/rabbitmq-operator/docs/examples/default-security-context/rabbitmq.yaml 2026-01-21 11:08:31.827388 | controller | RABBITMQ_DEPL_IMG: unused 2026-01-21 11:08:31.827394 | controller | RABBITMQ_IMG: quay.io/openstack-k8s-operators/rabbitmq-cluster-operator-index:latest 2026-01-21 11:08:31.827400 | controller | RABBITMQ_REPO: https://github.com/openstack-k8s-operators/rabbitmq-cluster-operator.git 2026-01-21 11:08:31.827407 | controller | REDHAT_OPERATORS: 'false' 2026-01-21 11:08:31.827413 | controller | REDIS: config/samples/redis_v1beta1_redis.yaml 2026-01-21 11:08:31.827419 | controller | REDIS_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/infra-operator-redis/config/samples/redis_v1beta1_redis.yaml 2026-01-21 11:08:31.827425 | controller | REDIS_DEPL_IMG: unused 2026-01-21 11:08:31.827432 | controller | RH_REGISTRY_PWD: '' 2026-01-21 11:08:31.827438 | controller | RH_REGISTRY_USER: '' 2026-01-21 11:08:31.827447 | controller | SECRET: osp-secret 2026-01-21 11:08:31.827454 | controller | SG_CORE_DEPL_IMG: unused 2026-01-21 11:08:31.827460 | controller | STANDALONE_COMPUTE_DRIVER: libvirt 2026-01-21 11:08:31.827466 | controller | STANDALONE_EXTERNAL_NET_PREFFIX: 172.21.0 2026-01-21 11:08:31.827472 | controller | STANDALONE_INTERNALAPI_NET_PREFIX: 172.17.0 2026-01-21 11:08:31.827479 | controller | STANDALONE_STORAGEMGMT_NET_PREFIX: 172.20.0 2026-01-21 11:08:31.827485 | controller | STANDALONE_STORAGE_NET_PREFIX: 172.18.0 2026-01-21 11:08:31.827491 | controller | STANDALONE_TENANT_NET_PREFIX: 172.19.0 2026-01-21 11:08:31.827497 | controller | STORAGEMGMT_HOST_ROUTES: '' 2026-01-21 11:08:31.827504 | controller | STORAGE_CLASS: local-storage 2026-01-21 11:08:31.827510 | controller | STORAGE_HOST_ROUTES: '' 2026-01-21 11:08:31.827516 | controller | SWIFT: config/samples/swift_v1beta1_swift.yaml 2026-01-21 11:08:31.827523 | controller | SWIFT_BRANCH: main 2026-01-21 11:08:31.827529 | controller | SWIFT_COMMIT_HASH: '' 2026-01-21 11:08:31.827535 | controller | SWIFT_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/swift-operator/config/samples/swift_v1beta1_swift.yaml 2026-01-21 11:08:31.827541 | controller | SWIFT_IMG: quay.io/openstack-k8s-operators/swift-operator-index:latest 2026-01-21 11:08:31.827548 | controller | SWIFT_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/swift-operator/kuttl-test.yaml 2026-01-21 11:08:31.827554 | controller | SWIFT_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/swift-operator/test/kuttl/tests 2026-01-21 11:08:31.827560 | controller | SWIFT_KUTTL_NAMESPACE: swift-kuttl-tests 2026-01-21 11:08:31.827566 | controller | SWIFT_REPO: https://github.com/openstack-k8s-operators/swift-operator.git 2026-01-21 11:08:31.827573 | controller | TELEMETRY: config/samples/telemetry_v1beta1_telemetry.yaml 2026-01-21 11:08:31.827579 | controller | TELEMETRY_BRANCH: main 2026-01-21 11:08:31.827585 | controller | TELEMETRY_COMMIT_HASH: '' 2026-01-21 11:08:31.827591 | controller | TELEMETRY_CR: /home/zuul/ci-framework-data/artifacts/manifests/operator/telemetry-operator/config/samples/telemetry_v1beta1_telemetry.yaml 2026-01-21 11:08:31.827598 | controller | TELEMETRY_IMG: quay.io/openstack-k8s-operators/telemetry-operator-index:latest 2026-01-21 11:08:31.827604 | controller | TELEMETRY_KUTTL_BASEDIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/telemetry-operator 2026-01-21 11:08:31.827610 | controller | TELEMETRY_KUTTL_CONF: /home/zuul/ci-framework-data/artifacts/manifests/operator/telemetry-operator/kuttl-test.yaml 2026-01-21 11:08:31.827617 | controller | TELEMETRY_KUTTL_DIR: /home/zuul/ci-framework-data/artifacts/manifests/operator/telemetry-operator/test/kuttl/suites 2026-01-21 11:08:31.827623 | controller | TELEMETRY_KUTTL_NAMESPACE: telemetry-kuttl-tests 2026-01-21 11:08:31.827629 | controller | TELEMETRY_KUTTL_RELPATH: test/kuttl/suites 2026-01-21 11:08:31.827636 | controller | TELEMETRY_REPO: https://github.com/openstack-k8s-operators/telemetry-operator.git 2026-01-21 11:08:31.827645 | controller | TENANT_HOST_ROUTES: '' 2026-01-21 11:08:31.841425 | controller | TIMEOUT: 300s 2026-01-21 11:08:31.841504 | controller | TLS_ENABLED: 'false' 2026-01-21 11:08:31.841512 | controller | WATCHER_BRANCH: '' 2026-01-21 11:08:31.841520 | controller | WATCHER_REPO: /home/zuul/src/github.com/openstack-k8s-operators/watcher-operator 2026-01-21 11:08:31.841527 | controller | tripleo_deploy: 'export REGISTRY_USER:' 2026-01-21 11:08:31.841546 | controller | 2026-01-21 11:08:32.251290 | controller | TASK [install_yamls : Generate make targets install_yamls_path={{ cifmw_install_yamls_repo }}, output_directory={{ cifmw_install_yamls_tasks_out }}] *** 2026-01-21 11:08:32.251524 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.049) 0:01:15.511 ***** 2026-01-21 11:08:32.251553 | controller | Wednesday 21 January 2026 11:08:31 +0000 (0:00:00.049) 0:01:15.510 ***** 2026-01-21 11:08:32.251587 | controller | ok: [controller] 2026-01-21 11:08:32.276251 | controller | 2026-01-21 11:08:32.276313 | controller | TASK [install_yamls : Debug generate_make module var=cifmw_generate_makes] ***** 2026-01-21 11:08:32.276332 | controller | Wednesday 21 January 2026 11:08:32 +0000 (0:00:00.410) 0:01:15.922 ***** 2026-01-21 11:08:32.276342 | controller | Wednesday 21 January 2026 11:08:32 +0000 (0:00:00.410) 0:01:15.921 ***** 2026-01-21 11:08:32.276358 | controller | ok: [controller] => 2026-01-21 11:08:32.276368 | controller | cifmw_generate_makes: 2026-01-21 11:08:32.276376 | controller | changed: false 2026-01-21 11:08:32.276384 | controller | debug: 2026-01-21 11:08:32.276393 | controller | /home/zuul/src/github.com/openstack-k8s-operators/install_yamls/Makefile: 2026-01-21 11:08:32.276401 | controller | - all 2026-01-21 11:08:32.276409 | controller | - help 2026-01-21 11:08:32.276417 | controller | - cleanup 2026-01-21 11:08:32.276426 | controller | - deploy_cleanup 2026-01-21 11:08:32.276435 | controller | - wait 2026-01-21 11:08:32.276443 | controller | - crc_storage 2026-01-21 11:08:32.276452 | controller | - crc_storage_cleanup 2026-01-21 11:08:32.276460 | controller | - crc_storage_release 2026-01-21 11:08:32.276469 | controller | - crc_storage_with_retries 2026-01-21 11:08:32.276478 | controller | - crc_storage_cleanup_with_retries 2026-01-21 11:08:32.276486 | controller | - operator_namespace 2026-01-21 11:08:32.276495 | controller | - namespace 2026-01-21 11:08:32.276504 | controller | - namespace_cleanup 2026-01-21 11:08:32.276513 | controller | - input 2026-01-21 11:08:32.276522 | controller | - input_cleanup 2026-01-21 11:08:32.276531 | controller | - crc_bmo_setup 2026-01-21 11:08:32.276541 | controller | - crc_bmo_cleanup 2026-01-21 11:08:32.276549 | controller | - openstack_prep 2026-01-21 11:08:32.276558 | controller | - openstack 2026-01-21 11:08:32.276566 | controller | - openstack_wait 2026-01-21 11:08:32.276574 | controller | - openstack_init 2026-01-21 11:08:32.276583 | controller | - openstack_cleanup 2026-01-21 11:08:32.276591 | controller | - openstack_repo 2026-01-21 11:08:32.276599 | controller | - openstack_deploy_prep 2026-01-21 11:08:32.276608 | controller | - openstack_deploy 2026-01-21 11:08:32.276625 | controller | - openstack_wait_deploy 2026-01-21 11:08:32.276635 | controller | - openstack_deploy_cleanup 2026-01-21 11:08:32.276644 | controller | - openstack_update_run 2026-01-21 11:08:32.276653 | controller | - update_services 2026-01-21 11:08:32.276661 | controller | - update_system 2026-01-21 11:08:32.276670 | controller | - openstack_patch_version 2026-01-21 11:08:32.276679 | controller | - edpm_deploy_generate_keys 2026-01-21 11:08:32.276688 | controller | - edpm_patch_ansible_runner_image 2026-01-21 11:08:32.276697 | controller | - edpm_deploy_prep 2026-01-21 11:08:32.276705 | controller | - edpm_deploy_cleanup 2026-01-21 11:08:32.276735 | controller | - edpm_deploy 2026-01-21 11:08:32.276751 | controller | - edpm_deploy_baremetal_prep 2026-01-21 11:08:32.276760 | controller | - edpm_deploy_baremetal 2026-01-21 11:08:32.276768 | controller | - edpm_wait_deploy_baremetal 2026-01-21 11:08:32.276777 | controller | - edpm_wait_deploy 2026-01-21 11:08:32.276786 | controller | - edpm_register_dns 2026-01-21 11:08:32.276794 | controller | - edpm_nova_discover_hosts 2026-01-21 11:08:32.276803 | controller | - openstack_crds 2026-01-21 11:08:32.276811 | controller | - openstack_crds_cleanup 2026-01-21 11:08:32.276820 | controller | - edpm_deploy_networker_prep 2026-01-21 11:08:32.276829 | controller | - edpm_deploy_networker_cleanup 2026-01-21 11:08:32.276837 | controller | - edpm_deploy_networker 2026-01-21 11:08:32.276846 | controller | - infra_prep 2026-01-21 11:08:32.276855 | controller | - infra 2026-01-21 11:08:32.276863 | controller | - infra_cleanup 2026-01-21 11:08:32.276872 | controller | - dns_deploy_prep 2026-01-21 11:08:32.276880 | controller | - dns_deploy 2026-01-21 11:08:32.276889 | controller | - dns_deploy_cleanup 2026-01-21 11:08:32.276904 | controller | - netconfig_deploy_prep 2026-01-21 11:08:32.276915 | controller | - netconfig_deploy 2026-01-21 11:08:32.276924 | controller | - netconfig_deploy_cleanup 2026-01-21 11:08:32.276932 | controller | - memcached_deploy_prep 2026-01-21 11:08:32.276940 | controller | - memcached_deploy 2026-01-21 11:08:32.276948 | controller | - memcached_deploy_cleanup 2026-01-21 11:08:32.276957 | controller | - keystone_prep 2026-01-21 11:08:32.276965 | controller | - keystone 2026-01-21 11:08:32.276973 | controller | - keystone_cleanup 2026-01-21 11:08:32.276982 | controller | - keystone_deploy_prep 2026-01-21 11:08:32.276991 | controller | - keystone_deploy 2026-01-21 11:08:32.277000 | controller | - keystone_deploy_cleanup 2026-01-21 11:08:32.277009 | controller | - barbican_prep 2026-01-21 11:08:32.277017 | controller | - barbican 2026-01-21 11:08:32.277026 | controller | - barbican_cleanup 2026-01-21 11:08:32.277034 | controller | - barbican_deploy_prep 2026-01-21 11:08:32.277043 | controller | - barbican_deploy 2026-01-21 11:08:32.277051 | controller | - barbican_deploy_validate 2026-01-21 11:08:32.277060 | controller | - barbican_deploy_cleanup 2026-01-21 11:08:32.277068 | controller | - mariadb 2026-01-21 11:08:32.277077 | controller | - mariadb_cleanup 2026-01-21 11:08:32.277091 | controller | - mariadb_deploy_prep 2026-01-21 11:08:32.277100 | controller | - mariadb_deploy 2026-01-21 11:08:32.277109 | controller | - mariadb_deploy_cleanup 2026-01-21 11:08:32.277117 | controller | - placement_prep 2026-01-21 11:08:32.277126 | controller | - placement 2026-01-21 11:08:32.277134 | controller | - placement_cleanup 2026-01-21 11:08:32.277143 | controller | - placement_deploy_prep 2026-01-21 11:08:32.277151 | controller | - placement_deploy 2026-01-21 11:08:32.277160 | controller | - placement_deploy_cleanup 2026-01-21 11:08:32.277168 | controller | - glance_prep 2026-01-21 11:08:32.277177 | controller | - glance 2026-01-21 11:08:32.277186 | controller | - glance_cleanup 2026-01-21 11:08:32.277194 | controller | - glance_deploy_prep 2026-01-21 11:08:32.277203 | controller | - glance_deploy 2026-01-21 11:08:32.277212 | controller | - glance_deploy_cleanup 2026-01-21 11:08:32.277220 | controller | - ovn_prep 2026-01-21 11:08:32.277229 | controller | - ovn 2026-01-21 11:08:32.277239 | controller | - ovn_cleanup 2026-01-21 11:08:32.277247 | controller | - ovn_deploy_prep 2026-01-21 11:08:32.277255 | controller | - ovn_deploy 2026-01-21 11:08:32.277264 | controller | - ovn_deploy_cleanup 2026-01-21 11:08:32.277272 | controller | - neutron_prep 2026-01-21 11:08:32.277280 | controller | - neutron 2026-01-21 11:08:32.277288 | controller | - neutron_cleanup 2026-01-21 11:08:32.277296 | controller | - neutron_deploy_prep 2026-01-21 11:08:32.277305 | controller | - neutron_deploy 2026-01-21 11:08:32.277313 | controller | - neutron_deploy_cleanup 2026-01-21 11:08:32.277322 | controller | - cinder_prep 2026-01-21 11:08:32.277331 | controller | - cinder 2026-01-21 11:08:32.277340 | controller | - cinder_cleanup 2026-01-21 11:08:32.277349 | controller | - cinder_deploy_prep 2026-01-21 11:08:32.277358 | controller | - cinder_deploy 2026-01-21 11:08:32.277367 | controller | - cinder_deploy_cleanup 2026-01-21 11:08:32.277376 | controller | - rabbitmq_prep 2026-01-21 11:08:32.277385 | controller | - rabbitmq 2026-01-21 11:08:32.277393 | controller | - rabbitmq_cleanup 2026-01-21 11:08:32.277402 | controller | - rabbitmq_deploy_prep 2026-01-21 11:08:32.277410 | controller | - rabbitmq_deploy 2026-01-21 11:08:32.277418 | controller | - rabbitmq_deploy_cleanup 2026-01-21 11:08:32.277427 | controller | - ironic_prep 2026-01-21 11:08:32.277436 | controller | - ironic 2026-01-21 11:08:32.277444 | controller | - ironic_cleanup 2026-01-21 11:08:32.277452 | controller | - ironic_deploy_prep 2026-01-21 11:08:32.277459 | controller | - ironic_deploy 2026-01-21 11:08:32.277467 | controller | - ironic_deploy_cleanup 2026-01-21 11:08:32.277475 | controller | - octavia_prep 2026-01-21 11:08:32.277483 | controller | - octavia 2026-01-21 11:08:32.277491 | controller | - octavia_cleanup 2026-01-21 11:08:32.277499 | controller | - octavia_deploy_prep 2026-01-21 11:08:32.277508 | controller | - octavia_deploy 2026-01-21 11:08:32.277519 | controller | - octavia_deploy_cleanup 2026-01-21 11:08:32.277528 | controller | - designate_prep 2026-01-21 11:08:32.277538 | controller | - designate 2026-01-21 11:08:32.277551 | controller | - designate_cleanup 2026-01-21 11:08:32.277561 | controller | - designate_deploy_prep 2026-01-21 11:08:32.277569 | controller | - designate_deploy 2026-01-21 11:08:32.277578 | controller | - designate_deploy_cleanup 2026-01-21 11:08:32.277586 | controller | - nova_prep 2026-01-21 11:08:32.277594 | controller | - nova 2026-01-21 11:08:32.277602 | controller | - nova_cleanup 2026-01-21 11:08:32.277611 | controller | - nova_deploy_prep 2026-01-21 11:08:32.277619 | controller | - nova_deploy 2026-01-21 11:08:32.277627 | controller | - nova_deploy_cleanup 2026-01-21 11:08:32.277635 | controller | - mariadb_kuttl_run 2026-01-21 11:08:32.277643 | controller | - mariadb_kuttl 2026-01-21 11:08:32.277652 | controller | - kuttl_db_prep 2026-01-21 11:08:32.277660 | controller | - kuttl_db_cleanup 2026-01-21 11:08:32.277668 | controller | - kuttl_common_prep 2026-01-21 11:08:32.277676 | controller | - kuttl_common_cleanup 2026-01-21 11:08:32.277685 | controller | - keystone_kuttl_run 2026-01-21 11:08:32.277693 | controller | - keystone_kuttl 2026-01-21 11:08:32.277701 | controller | - barbican_kuttl_run 2026-01-21 11:08:32.277726 | controller | - barbican_kuttl 2026-01-21 11:08:32.277748 | controller | - placement_kuttl_run 2026-01-21 11:08:32.277758 | controller | - placement_kuttl 2026-01-21 11:08:32.277767 | controller | - cinder_kuttl_run 2026-01-21 11:08:32.277783 | controller | - cinder_kuttl 2026-01-21 11:08:32.277792 | controller | - neutron_kuttl_run 2026-01-21 11:08:32.277801 | controller | - neutron_kuttl 2026-01-21 11:08:32.277810 | controller | - octavia_kuttl_run 2026-01-21 11:08:32.277819 | controller | - octavia_kuttl 2026-01-21 11:08:32.277828 | controller | - designate_kuttl 2026-01-21 11:08:32.277837 | controller | - designate_kuttl_run 2026-01-21 11:08:32.277845 | controller | - ovn_kuttl_run 2026-01-21 11:08:32.277853 | controller | - ovn_kuttl 2026-01-21 11:08:32.277861 | controller | - infra_kuttl_run 2026-01-21 11:08:32.277869 | controller | - infra_kuttl 2026-01-21 11:08:32.277877 | controller | - ironic_kuttl_run 2026-01-21 11:08:32.277885 | controller | - ironic_kuttl 2026-01-21 11:08:32.277893 | controller | - ironic_kuttl_crc 2026-01-21 11:08:32.277901 | controller | - heat_kuttl_run 2026-01-21 11:08:32.277910 | controller | - heat_kuttl 2026-01-21 11:08:32.277918 | controller | - heat_kuttl_crc 2026-01-21 11:08:32.277926 | controller | - ansibleee_kuttl_run 2026-01-21 11:08:32.277935 | controller | - ansibleee_kuttl_cleanup 2026-01-21 11:08:32.277943 | controller | - ansibleee_kuttl_prep 2026-01-21 11:08:32.277952 | controller | - ansibleee_kuttl 2026-01-21 11:08:32.277960 | controller | - glance_kuttl_run 2026-01-21 11:08:32.277969 | controller | - glance_kuttl 2026-01-21 11:08:32.277977 | controller | - manila_kuttl_run 2026-01-21 11:08:32.277985 | controller | - manila_kuttl 2026-01-21 11:08:32.277994 | controller | - swift_kuttl_run 2026-01-21 11:08:32.278002 | controller | - swift_kuttl 2026-01-21 11:08:32.278010 | controller | - horizon_kuttl_run 2026-01-21 11:08:32.278018 | controller | - horizon_kuttl 2026-01-21 11:08:32.278026 | controller | - openstack_kuttl_run 2026-01-21 11:08:32.278035 | controller | - openstack_kuttl 2026-01-21 11:08:32.278043 | controller | - mariadb_chainsaw_run 2026-01-21 11:08:32.278051 | controller | - mariadb_chainsaw 2026-01-21 11:08:32.278060 | controller | - horizon_prep 2026-01-21 11:08:32.278068 | controller | - horizon 2026-01-21 11:08:32.278077 | controller | - horizon_cleanup 2026-01-21 11:08:32.278085 | controller | - horizon_deploy_prep 2026-01-21 11:08:32.278094 | controller | - horizon_deploy 2026-01-21 11:08:32.278103 | controller | - horizon_deploy_cleanup 2026-01-21 11:08:32.278118 | controller | - heat_prep 2026-01-21 11:08:32.278126 | controller | - heat 2026-01-21 11:08:32.278135 | controller | - heat_cleanup 2026-01-21 11:08:32.278144 | controller | - heat_deploy_prep 2026-01-21 11:08:32.278152 | controller | - heat_deploy 2026-01-21 11:08:32.278160 | controller | - heat_deploy_cleanup 2026-01-21 11:08:32.278169 | controller | - ansibleee_prep 2026-01-21 11:08:32.278176 | controller | - ansibleee 2026-01-21 11:08:32.278184 | controller | - ansibleee_cleanup 2026-01-21 11:08:32.278192 | controller | - baremetal_prep 2026-01-21 11:08:32.278200 | controller | - baremetal 2026-01-21 11:08:32.278208 | controller | - baremetal_cleanup 2026-01-21 11:08:32.278217 | controller | - ceph_help 2026-01-21 11:08:32.278230 | controller | - ceph 2026-01-21 11:08:32.278240 | controller | - ceph_cleanup 2026-01-21 11:08:32.278248 | controller | - rook_prep 2026-01-21 11:08:32.278256 | controller | - rook 2026-01-21 11:08:32.278264 | controller | - rook_deploy_prep 2026-01-21 11:08:32.278273 | controller | - rook_deploy 2026-01-21 11:08:32.278281 | controller | - rook_crc_disk 2026-01-21 11:08:32.278289 | controller | - rook_cleanup 2026-01-21 11:08:32.278297 | controller | - lvms 2026-01-21 11:08:32.278305 | controller | - nmstate 2026-01-21 11:08:32.278313 | controller | - nncp 2026-01-21 11:08:32.278322 | controller | - nncp_cleanup 2026-01-21 11:08:32.278330 | controller | - netattach 2026-01-21 11:08:32.278338 | controller | - netattach_cleanup 2026-01-21 11:08:32.278346 | controller | - metallb 2026-01-21 11:08:32.278354 | controller | - metallb_config 2026-01-21 11:08:32.278363 | controller | - metallb_config_cleanup 2026-01-21 11:08:32.278371 | controller | - metallb_cleanup 2026-01-21 11:08:32.278380 | controller | - loki 2026-01-21 11:08:32.278388 | controller | - loki_cleanup 2026-01-21 11:08:32.278400 | controller | - loki_deploy 2026-01-21 11:08:32.278409 | controller | - loki_deploy_cleanup 2026-01-21 11:08:32.278418 | controller | - netobserv 2026-01-21 11:08:32.278426 | controller | - netobserv_cleanup 2026-01-21 11:08:32.278435 | controller | - netobserv_deploy 2026-01-21 11:08:32.278444 | controller | - netobserv_deploy_cleanup 2026-01-21 11:08:32.278453 | controller | - manila_prep 2026-01-21 11:08:32.278462 | controller | - manila 2026-01-21 11:08:32.278471 | controller | - manila_cleanup 2026-01-21 11:08:32.278479 | controller | - manila_deploy_prep 2026-01-21 11:08:32.278487 | controller | - manila_deploy 2026-01-21 11:08:32.278497 | controller | - manila_deploy_cleanup 2026-01-21 11:08:32.278506 | controller | - telemetry_prep 2026-01-21 11:08:32.278515 | controller | - telemetry 2026-01-21 11:08:32.278523 | controller | - telemetry_cleanup 2026-01-21 11:08:32.278532 | controller | - telemetry_deploy_prep 2026-01-21 11:08:32.278541 | controller | - telemetry_deploy 2026-01-21 11:08:32.278550 | controller | - telemetry_deploy_cleanup 2026-01-21 11:08:32.278559 | controller | - telemetry_kuttl_run 2026-01-21 11:08:32.278567 | controller | - telemetry_kuttl 2026-01-21 11:08:32.278575 | controller | - swift_prep 2026-01-21 11:08:32.278583 | controller | - swift 2026-01-21 11:08:32.278591 | controller | - swift_cleanup 2026-01-21 11:08:32.278599 | controller | - swift_deploy_prep 2026-01-21 11:08:32.278607 | controller | - swift_deploy 2026-01-21 11:08:32.278614 | controller | - swift_deploy_cleanup 2026-01-21 11:08:32.278623 | controller | - certmanager 2026-01-21 11:08:32.278631 | controller | - certmanager_cleanup 2026-01-21 11:08:32.278639 | controller | - validate_marketplace 2026-01-21 11:08:32.278652 | controller | - redis_deploy_prep 2026-01-21 11:08:32.278662 | controller | - redis_deploy 2026-01-21 11:08:32.278671 | controller | - redis_deploy_cleanup 2026-01-21 11:08:32.278679 | controller | - set_slower_etcd_profile 2026-01-21 11:08:32.278687 | controller | /home/zuul/src/github.com/openstack-k8s-operators/install_yamls/devsetup/Makefile: 2026-01-21 11:08:32.278696 | controller | - help 2026-01-21 11:08:32.278705 | controller | - download_tools 2026-01-21 11:08:32.278734 | controller | - nfs 2026-01-21 11:08:32.278750 | controller | - nfs_cleanup 2026-01-21 11:08:32.278759 | controller | - crc 2026-01-21 11:08:32.278768 | controller | - crc_cleanup 2026-01-21 11:08:32.278776 | controller | - crc_scrub 2026-01-21 11:08:32.278784 | controller | - crc_attach_default_interface 2026-01-21 11:08:32.278792 | controller | - crc_attach_default_interface_cleanup 2026-01-21 11:08:32.278801 | controller | - ipv6_lab_network 2026-01-21 11:08:32.278809 | controller | - ipv6_lab_network_cleanup 2026-01-21 11:08:32.278818 | controller | - ipv6_lab_nat64_router 2026-01-21 11:08:32.278827 | controller | - ipv6_lab_nat64_router_cleanup 2026-01-21 11:08:32.278836 | controller | - ipv6_lab_sno 2026-01-21 11:08:32.278845 | controller | - ipv6_lab_sno_cleanup 2026-01-21 11:08:32.278853 | controller | - ipv6_lab 2026-01-21 11:08:32.278862 | controller | - ipv6_lab_cleanup 2026-01-21 11:08:32.278871 | controller | - attach_default_interface 2026-01-21 11:08:32.278880 | controller | - attach_default_interface_cleanup 2026-01-21 11:08:32.278889 | controller | - network_isolation_bridge 2026-01-21 11:08:32.278902 | controller | - network_isolation_bridge_cleanup 2026-01-21 11:08:32.278911 | controller | - edpm_baremetal_compute 2026-01-21 11:08:32.278920 | controller | - edpm_compute 2026-01-21 11:08:32.278929 | controller | - edpm_compute_bootc 2026-01-21 11:08:32.278938 | controller | - edpm_ansible_runner 2026-01-21 11:08:32.278947 | controller | - edpm_computes_bgp 2026-01-21 11:08:32.278956 | controller | - edpm_compute_repos 2026-01-21 11:08:32.278982 | controller | - edpm_compute_cleanup 2026-01-21 11:08:32.278993 | controller | - edpm_networker 2026-01-21 11:08:32.279003 | controller | - edpm_networker_cleanup 2026-01-21 11:08:32.279012 | controller | - edpm_deploy_instance 2026-01-21 11:08:32.279021 | controller | - tripleo_deploy 2026-01-21 11:08:32.279030 | controller | - standalone_deploy 2026-01-21 11:08:32.279040 | controller | - standalone_sync 2026-01-21 11:08:32.279049 | controller | - standalone 2026-01-21 11:08:32.279058 | controller | - standalone_cleanup 2026-01-21 11:08:32.279067 | controller | - standalone_snapshot 2026-01-21 11:08:32.279076 | controller | - standalone_revert 2026-01-21 11:08:32.279086 | controller | - cifmw_prepare 2026-01-21 11:08:32.279095 | controller | - cifmw_cleanup 2026-01-21 11:08:32.279105 | controller | - bmaas_network 2026-01-21 11:08:32.279114 | controller | - bmaas_network_cleanup 2026-01-21 11:08:32.279124 | controller | - bmaas_route_crc_and_crc_bmaas_networks 2026-01-21 11:08:32.279133 | controller | - bmaas_route_crc_and_crc_bmaas_networks_cleanup 2026-01-21 11:08:32.279149 | controller | - bmaas_crc_attach_network 2026-01-21 11:08:32.286664 | controller | - bmaas_crc_attach_network_cleanup 2026-01-21 11:08:32.286741 | controller | - bmaas_crc_baremetal_bridge 2026-01-21 11:08:32.286757 | controller | - bmaas_crc_baremetal_bridge_cleanup 2026-01-21 11:08:32.286765 | controller | - bmaas_baremetal_net_nad 2026-01-21 11:08:32.286773 | controller | - bmaas_baremetal_net_nad_cleanup 2026-01-21 11:08:32.286781 | controller | - bmaas_metallb 2026-01-21 11:08:32.286788 | controller | - bmaas_metallb_cleanup 2026-01-21 11:08:32.286796 | controller | - bmaas_virtual_bms 2026-01-21 11:08:32.286803 | controller | - bmaas_virtual_bms_cleanup 2026-01-21 11:08:32.286811 | controller | - bmaas_sushy_emulator 2026-01-21 11:08:32.286818 | controller | - bmaas_sushy_emulator_cleanup 2026-01-21 11:08:32.286826 | controller | - bmaas_sushy_emulator_wait 2026-01-21 11:08:32.286833 | controller | - bmaas_generate_nodes_yaml 2026-01-21 11:08:32.286840 | controller | - bmaas 2026-01-21 11:08:32.286848 | controller | - bmaas_cleanup 2026-01-21 11:08:32.286855 | controller | failed: false 2026-01-21 11:08:32.286863 | controller | success: true 2026-01-21 11:08:32.286878 | controller | 2026-01-21 11:08:32.967786 | controller | TASK [install_yamls : Create the install_yamls parameters file dest={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts/parameters/install-yamls-params.yml, content={{ 2026-01-21 11:08:32.967863 | controller | { 2026-01-21 11:08:32.967873 | controller | 'cifmw_install_yamls_environment': cifmw_install_yamls_environment, 2026-01-21 11:08:32.967882 | controller | 'cifmw_install_yamls_defaults': cifmw_install_yamls_defaults 2026-01-21 11:08:32.967888 | controller | } | to_nice_yaml 2026-01-21 11:08:32.967895 | controller | }}, mode=0644] *** 2026-01-21 11:08:32.967902 | controller | Wednesday 21 January 2026 11:08:32 +0000 (0:00:00.035) 0:01:15.958 ***** 2026-01-21 11:08:32.967908 | controller | Wednesday 21 January 2026 11:08:32 +0000 (0:00:00.035) 0:01:15.956 ***** 2026-01-21 11:08:32.967921 | controller | changed: [controller] 2026-01-21 11:08:33.006333 | controller | 2026-01-21 11:08:33.006383 | controller | TASK [install_yamls : Create empty cifmw_install_yamls_environment if needed cifmw_install_yamls_environment={}] *** 2026-01-21 11:08:33.006394 | controller | Wednesday 21 January 2026 11:08:32 +0000 (0:00:00.680) 0:01:16.639 ***** 2026-01-21 11:08:33.006402 | controller | Wednesday 21 January 2026 11:08:32 +0000 (0:00:00.680) 0:01:16.637 ***** 2026-01-21 11:08:33.006414 | controller | skipping: [controller] 2026-01-21 11:08:33.443561 | controller | 2026-01-21 11:08:33.443604 | controller | TASK [discover_latest_image : Get latest image url={{ cifmw_discover_latest_image_base_url }}, image_prefix={{ cifmw_discover_latest_image_qcow_prefix }}, images_file={{ cifmw_discover_latest_image_images_file }}] *** 2026-01-21 11:08:33.443621 | controller | Wednesday 21 January 2026 11:08:33 +0000 (0:00:00.038) 0:01:16.677 ***** 2026-01-21 11:08:33.443631 | controller | Wednesday 21 January 2026 11:08:33 +0000 (0:00:00.038) 0:01:16.676 ***** 2026-01-21 11:08:33.443643 | controller | changed: [controller] 2026-01-21 11:08:33.481180 | controller | 2026-01-21 11:08:33.481222 | controller | TASK [discover_latest_image : Export facts accordingly cifmw_discovered_image_name={{ discovered_image['data']['image_name'] }}, cifmw_discovered_image_url={{ discovered_image['data']['image_url'] }}, cifmw_discovered_hash={{ discovered_image['data']['hash'] }}, cifmw_discovered_hash_algorithm={{ discovered_image['data']['hash_algorithm'] }}, cacheable=True] *** 2026-01-21 11:08:33.481230 | controller | Wednesday 21 January 2026 11:08:33 +0000 (0:00:00.437) 0:01:17.115 ***** 2026-01-21 11:08:33.481238 | controller | Wednesday 21 January 2026 11:08:33 +0000 (0:00:00.437) 0:01:17.113 ***** 2026-01-21 11:08:33.481249 | controller | ok: [controller] 2026-01-21 11:08:34.262319 | controller | 2026-01-21 11:08:34.262365 | controller | TASK [cifmw_setup : Create artifacts with custom params mode=0644, dest={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts/parameters/custom-params.yml, content={{ ci_framework_params | to_nice_yaml }}] *** 2026-01-21 11:08:34.262376 | controller | Wednesday 21 January 2026 11:08:33 +0000 (0:00:00.037) 0:01:17.152 ***** 2026-01-21 11:08:34.262385 | controller | Wednesday 21 January 2026 11:08:33 +0000 (0:00:00.037) 0:01:17.151 ***** 2026-01-21 11:08:34.262399 | controller | changed: [controller] 2026-01-21 11:08:34.298189 | controller | 2026-01-21 11:08:34.298230 | controller | PLAY [Install dev tools] ******************************************************* 2026-01-21 11:08:34.298239 | controller | 2026-01-21 11:08:34.298246 | controller | TASK [Assert that operator_name is set that=['operator_name is defined']] ****** 2026-01-21 11:08:34.298252 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.781) 0:01:17.933 ***** 2026-01-21 11:08:34.298258 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.781) 0:01:17.932 ***** 2026-01-21 11:08:34.298270 | controller | ok: [controller] => 2026-01-21 11:08:34.333774 | controller | changed: false 2026-01-21 11:08:34.333813 | controller | msg: All assertions passed 2026-01-21 11:08:34.333821 | controller | 2026-01-21 11:08:34.333829 | controller | TASK [Download install_yamls deps name=install_yamls_makes, tasks_from=make_download_tools] *** 2026-01-21 11:08:34.333836 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.035) 0:01:17.969 ***** 2026-01-21 11:08:34.333842 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.035) 0:01:17.968 ***** 2026-01-21 11:08:34.333854 | controller | 2026-01-21 11:08:34.366681 | controller | TASK [install_yamls_makes : Debug make_download_tools_env var=make_download_tools_env] *** 2026-01-21 11:08:34.366751 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.035) 0:01:18.005 ***** 2026-01-21 11:08:34.366763 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.035) 0:01:18.003 ***** 2026-01-21 11:08:34.366776 | controller | skipping: [controller] 2026-01-21 11:08:34.396397 | controller | 2026-01-21 11:08:34.396436 | controller | TASK [install_yamls_makes : Debug make_download_tools_params var=make_download_tools_params] *** 2026-01-21 11:08:34.396447 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.032) 0:01:18.038 ***** 2026-01-21 11:08:34.396455 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.032) 0:01:18.036 ***** 2026-01-21 11:08:34.396470 | controller | skipping: [controller] 2026-01-21 11:09:13.861589 | controller | 2026-01-21 11:09:13.861849 | controller | TASK [install_yamls_makes : Run download_tools output_dir={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts, chdir=/home/zuul/src/github.com/openstack-k8s-operators/install_yamls/devsetup, script=make download_tools, dry_run={{ make_download_tools_dryrun|default(false)|bool }}, extra_args={{ dict((make_download_tools_env|default({})), **(make_download_tools_params|default({}))) }}] *** 2026-01-21 11:09:13.861864 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.029) 0:01:18.068 ***** 2026-01-21 11:09:13.861886 | controller | Wednesday 21 January 2026 11:08:34 +0000 (0:00:00.029) 0:01:18.066 ***** 2026-01-21 11:09:13.861924 | controller | [WARNING]: conditional statements should not include jinja2 templating 2026-01-21 11:09:13.892296 | controller | delimiters such as {{ }} or {% %}. Found: {{ make_download_tools_until | 2026-01-21 11:09:13.892417 | controller | default(true) }} 2026-01-21 11:09:13.892444 | controller | Follow script's output here: /home/zuul/ci-framework-data/logs/ci_script_000_run_download.log 2026-01-21 11:09:13.981075 | controller | changed: [controller] 2026-01-21 11:09:13.981132 | controller | 2026-01-21 11:09:13.981143 | controller | TASK [run_hook : Assert parameters are valid quiet=True, that=['_list_hooks is not string', '_list_hooks is not mapping', '_list_hooks is iterable', '(hooks | default([])) is not string', '(hooks | default([])) is not mapping', '(hooks | default([])) is iterable']] *** 2026-01-21 11:09:13.981160 | controller | Wednesday 21 January 2026 11:09:13 +0000 (0:00:39.494) 0:01:57.562 ***** 2026-01-21 11:09:13.981169 | controller | Wednesday 21 January 2026 11:09:13 +0000 (0:00:39.494) 0:01:57.561 ***** 2026-01-21 11:09:13.981185 | controller | ok: [controller] 2026-01-21 11:09:14.097480 | controller | 2026-01-21 11:09:14.097573 | controller | TASK [run_hook : Assert single hooks are all mappings quiet=True, that=['_not_mapping_hooks | length == 0'], msg=All single hooks must be a list of mappings or a mapping.] *** 2026-01-21 11:09:14.097590 | controller | Wednesday 21 January 2026 11:09:13 +0000 (0:00:00.089) 0:01:57.652 ***** 2026-01-21 11:09:14.097609 | controller | Wednesday 21 January 2026 11:09:13 +0000 (0:00:00.089) 0:01:57.651 ***** 2026-01-21 11:09:14.097625 | controller | ok: [controller] 2026-01-21 11:09:14.228470 | controller | 2026-01-21 11:09:14.228504 | controller | TASK [run_hook : Loop on hooks for pre_infra _raw_params={{ hook.type }}.yml] *** 2026-01-21 11:09:14.228513 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.116) 0:01:57.768 ***** 2026-01-21 11:09:14.228520 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.116) 0:01:57.767 ***** 2026-01-21 11:09:14.228540 | controller | included: /home/zuul/src/github.com/openstack-k8s-operators/ci-framework/roles/run_hook/tasks/playbook.yml for controller => (item={'name': 'Download needed tools', 'inventory': 'localhost,', 'connection': 'local', 'type': 'playbook', 'source': '/home/zuul/src/github.com/openstack-k8s-operators/install_yamls/devsetup/download_tools.yaml'}) 2026-01-21 11:09:14.286745 | controller | 2026-01-21 11:09:14.286804 | controller | TASK [run_hook : Set playbook path for Download needed tools cifmw_basedir={{ _bdir }}, hook_name={{ _hook_name }}, playbook_path={{ _play | realpath }}, log_path={{ _bdir }}/logs/{{ step }}_{{ _hook_name }}.log, extra_vars=-e namespace={{ cifmw_openstack_namespace }} {%- if hook.extra_vars is defined and hook.extra_vars|length > 0 -%} {% for key,value in hook.extra_vars.items() -%} {%- if key == 'file' %} -e "@{{ value }}" {%- else %} -e "{{ key }}={{ value }}" {%- endif %} {%- endfor %} {%- endif %}] *** 2026-01-21 11:09:14.286831 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.131) 0:01:57.900 ***** 2026-01-21 11:09:14.286841 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.131) 0:01:57.898 ***** 2026-01-21 11:09:14.286856 | controller | ok: [controller] 2026-01-21 11:09:14.550050 | controller | 2026-01-21 11:09:14.550113 | controller | TASK [run_hook : Get file stat path={{ playbook_path }}] *********************** 2026-01-21 11:09:14.550123 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.057) 0:01:57.958 ***** 2026-01-21 11:09:14.550130 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.057) 0:01:57.956 ***** 2026-01-21 11:09:14.550142 | controller | ok: [controller] 2026-01-21 11:09:14.573189 | controller | 2026-01-21 11:09:14.573229 | controller | TASK [run_hook : Fail if playbook doesn't exist msg=Playbook {{ playbook_path }} doesn't seem to exist.] *** 2026-01-21 11:09:14.573248 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.263) 0:01:58.221 ***** 2026-01-21 11:09:14.573255 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.263) 0:01:58.220 ***** 2026-01-21 11:09:14.573267 | controller | skipping: [controller] 2026-01-21 11:09:14.830737 | controller | 2026-01-21 11:09:14.830796 | controller | TASK [run_hook : Get parameters files paths={{ 2026-01-21 11:09:14.830823 | controller | (cifmw_basedir, 'artifacts/parameters') | path_join 2026-01-21 11:09:14.830834 | controller | }}, file_type=file, patterns=*.yml] *** 2026-01-21 11:09:14.830843 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.023) 0:01:58.244 ***** 2026-01-21 11:09:14.830852 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.023) 0:01:58.243 ***** 2026-01-21 11:09:14.830868 | controller | ok: [controller] 2026-01-21 11:09:14.860245 | controller | 2026-01-21 11:09:14.860329 | controller | TASK [run_hook : Add parameters artifacts as extra variables extra_vars={{ extra_vars }} {% for file in cifmw_run_hook_parameters_files.files %} -e "@{{ file.path }}" {%- endfor %}] *** 2026-01-21 11:09:14.860341 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.256) 0:01:58.501 ***** 2026-01-21 11:09:14.860348 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.257) 0:01:58.500 ***** 2026-01-21 11:09:14.860361 | controller | ok: [controller] 2026-01-21 11:09:15.150993 | controller | 2026-01-21 11:09:15.151034 | controller | TASK [run_hook : Ensure log directory exists path={{ log_path | dirname }}, state=directory, mode=0755] *** 2026-01-21 11:09:15.151043 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.029) 0:01:58.531 ***** 2026-01-21 11:09:15.151050 | controller | Wednesday 21 January 2026 11:09:14 +0000 (0:00:00.029) 0:01:58.530 ***** 2026-01-21 11:09:15.151061 | controller | ok: [controller] 2026-01-21 11:09:15.466887 | controller | 2026-01-21 11:09:15.466940 | controller | TASK [run_hook : Ensure artifacts directory exists path={{ cifmw_basedir }}/artifacts, state=directory, mode=0755] *** 2026-01-21 11:09:15.466949 | controller | Wednesday 21 January 2026 11:09:15 +0000 (0:00:00.290) 0:01:58.822 ***** 2026-01-21 11:09:15.466956 | controller | Wednesday 21 January 2026 11:09:15 +0000 (0:00:00.290) 0:01:58.821 ***** 2026-01-21 11:09:15.466969 | controller | ok: [controller] 2026-01-21 11:09:49.593810 | controller | 2026-01-21 11:09:49.594029 | controller | TASK [run_hook : Run hook without retry - Download needed tools] *************** 2026-01-21 11:09:49.594050 | controller | Wednesday 21 January 2026 11:09:15 +0000 (0:00:00.315) 0:01:59.138 ***** 2026-01-21 11:09:49.594059 | controller | Wednesday 21 January 2026 11:09:15 +0000 (0:00:00.315) 0:01:59.136 ***** 2026-01-21 11:09:49.594083 | controller | Follow script's output here: /home/zuul/ci-framework-data/logs/ci_script_001_run_hook_without_retry.log 2026-01-21 11:09:49.631576 | controller | changed: [controller] 2026-01-21 11:09:49.631635 | controller | 2026-01-21 11:09:49.631645 | controller | TASK [run_hook : Run hook with retry - Download needed tools] ****************** 2026-01-21 11:09:49.631654 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:34.125) 0:02:33.263 ***** 2026-01-21 11:09:49.631662 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:34.125) 0:02:33.262 ***** 2026-01-21 11:09:49.631677 | controller | skipping: [controller] 2026-01-21 11:09:49.883806 | controller | 2026-01-21 11:09:49.883864 | controller | TASK [run_hook : Check if we have a file path={{ cifmw_basedir }}/artifacts/{{ step }}_{{ hook_name }}.yml] *** 2026-01-21 11:09:49.883875 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:00.039) 0:02:33.302 ***** 2026-01-21 11:09:49.883884 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:00.039) 0:02:33.301 ***** 2026-01-21 11:09:49.883899 | controller | ok: [controller] 2026-01-21 11:09:49.948234 | controller | 2026-01-21 11:09:49.948278 | controller | TASK [run_hook : Load generated content in main playbook file={{ cifmw_basedir }}/artifacts/{{ step }}_{{ hook_name }}.yml] *** 2026-01-21 11:09:49.948287 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:00.252) 0:02:33.555 ***** 2026-01-21 11:09:49.948298 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:00.252) 0:02:33.554 ***** 2026-01-21 11:09:49.948310 | controller | skipping: [controller] 2026-01-21 11:09:50.004182 | controller | 2026-01-21 11:09:50.004234 | controller | PLAY [Prepare host virtualization] ********************************************* 2026-01-21 11:09:50.004247 | controller | 2026-01-21 11:09:50.004257 | controller | TASK [cifmw_setup : Load parameters files dir={{ cifmw_basedir }}/artifacts/parameters] *** 2026-01-21 11:09:50.004273 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:00.064) 0:02:33.619 ***** 2026-01-21 11:09:50.004283 | controller | Wednesday 21 January 2026 11:09:49 +0000 (0:00:00.064) 0:02:33.618 ***** 2026-01-21 11:09:50.004308 | controller | ok: [controller] 2026-01-21 11:09:50.043758 | controller | 2026-01-21 11:09:50.043804 | controller | TASK [Ensure libvirt is present/configured name=libvirt_manager] *************** 2026-01-21 11:09:50.043813 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.055) 0:02:33.675 ***** 2026-01-21 11:09:50.043820 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.055) 0:02:33.674 ***** 2026-01-21 11:09:50.043833 | controller | skipping: [controller] 2026-01-21 11:09:50.131837 | controller | 2026-01-21 11:09:50.131932 | controller | TASK [Perpare OpenShift provisioner node name=openshift_provisioner_node] ****** 2026-01-21 11:09:50.131960 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.039) 0:02:33.715 ***** 2026-01-21 11:09:50.131975 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.039) 0:02:33.713 ***** 2026-01-21 11:09:50.132006 | controller | skipping: [controller] 2026-01-21 11:09:50.246665 | controller | 2026-01-21 11:09:50.246738 | controller | PLAY [Build dataset hook] ****************************************************** 2026-01-21 11:09:50.246756 | controller | 2026-01-21 11:09:50.246765 | controller | TASK [cifmw_setup : Load parameters files dir={{ cifmw_basedir }}/artifacts/parameters] *** 2026-01-21 11:09:50.246775 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.088) 0:02:33.803 ***** 2026-01-21 11:09:50.246785 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.088) 0:02:33.802 ***** 2026-01-21 11:09:50.246803 | controller | ok: [controller] 2026-01-21 11:09:51.852004 | controller | 2026-01-21 11:09:51.852051 | controller | TASK [networking_mapper : Check for Networking Environment Definition file existence path={{ cifmw_networking_mapper_networking_env_def_path }}] *** 2026-01-21 11:09:51.852070 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.114) 0:02:33.917 ***** 2026-01-21 11:09:51.852079 | controller | Wednesday 21 January 2026 11:09:50 +0000 (0:00:00.114) 0:02:33.916 ***** 2026-01-21 11:09:51.852094 | controller | ok: [controller] 2026-01-21 11:09:51.885068 | controller | 2026-01-21 11:09:51.885113 | controller | TASK [networking_mapper : Check for Networking Definition file existance that=['_net_env_def_stat.stat.exists'], msg=Ensure that the Networking Environment Definition file exists in {{ cifmw_networking_mapper_networking_env_def_path }}, quiet=True] *** 2026-01-21 11:09:51.885140 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:01.605) 0:02:35.523 ***** 2026-01-21 11:09:51.885150 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:01.605) 0:02:35.521 ***** 2026-01-21 11:09:51.885164 | controller | skipping: [controller] 2026-01-21 11:09:51.915740 | controller | 2026-01-21 11:09:51.915786 | controller | TASK [networking_mapper : Load the Networking Definition from file path={{ cifmw_networking_mapper_networking_env_def_path }}] *** 2026-01-21 11:09:51.915806 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.033) 0:02:35.556 ***** 2026-01-21 11:09:51.915813 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.033) 0:02:35.555 ***** 2026-01-21 11:09:51.915825 | controller | skipping: [controller] 2026-01-21 11:09:51.953037 | controller | 2026-01-21 11:09:51.953260 | controller | TASK [networking_mapper : Set cifmw_networking_env_definition is present cifmw_networking_env_definition={{ 2026-01-21 11:09:51.953270 | controller | _net_env_def_slurp['content'] | 2026-01-21 11:09:51.953279 | controller | b64decode | 2026-01-21 11:09:51.953286 | controller | from_yaml 2026-01-21 11:09:51.953294 | controller | }}, cacheable=True] *** 2026-01-21 11:09:51.953302 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.030) 0:02:35.587 ***** 2026-01-21 11:09:51.953310 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.030) 0:02:35.585 ***** 2026-01-21 11:09:51.953324 | controller | skipping: [controller] 2026-01-21 11:09:51.982541 | controller | 2026-01-21 11:09:51.982580 | controller | TASK [Deploy OCP using Hive name=hive] ***************************************** 2026-01-21 11:09:51.982589 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.037) 0:02:35.624 ***** 2026-01-21 11:09:51.982596 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.037) 0:02:35.623 ***** 2026-01-21 11:09:51.982608 | controller | skipping: [controller] 2026-01-21 11:09:52.013133 | controller | 2026-01-21 11:09:52.013183 | controller | TASK [Prepare CRC name=rhol_crc] *********************************************** 2026-01-21 11:09:52.013193 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.029) 0:02:35.654 ***** 2026-01-21 11:09:52.013201 | controller | Wednesday 21 January 2026 11:09:51 +0000 (0:00:00.029) 0:02:35.652 ***** 2026-01-21 11:09:52.013215 | controller | skipping: [controller] 2026-01-21 11:09:52.045838 | controller | 2026-01-21 11:09:52.045880 | controller | TASK [Deploy OpenShift cluster using dev-scripts name=devscripts] ************** 2026-01-21 11:09:52.045889 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.030) 0:02:35.684 ***** 2026-01-21 11:09:52.045896 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.030) 0:02:35.683 ***** 2026-01-21 11:09:52.045907 | controller | skipping: [controller] 2026-01-21 11:09:52.312216 | controller | 2026-01-21 11:09:52.312329 | controller | TASK [openshift_login : Ensure output directory exists path={{ cifmw_openshift_login_basedir }}/artifacts, state=directory, mode=0755] *** 2026-01-21 11:09:52.312348 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.032) 0:02:35.717 ***** 2026-01-21 11:09:52.312360 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.032) 0:02:35.716 ***** 2026-01-21 11:09:52.312389 | controller | ok: [controller] 2026-01-21 11:09:52.357654 | controller | 2026-01-21 11:09:52.357697 | controller | TASK [openshift_login : OpenShift login _raw_params=login.yml] ***************** 2026-01-21 11:09:52.357706 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.265) 0:02:35.983 ***** 2026-01-21 11:09:52.357739 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.265) 0:02:35.982 ***** 2026-01-21 11:09:52.357770 | controller | included: /home/zuul/src/github.com/openstack-k8s-operators/ci-framework/roles/openshift_login/tasks/login.yml for controller 2026-01-21 11:09:52.389634 | controller | 2026-01-21 11:09:52.389676 | controller | TASK [openshift_login : Check if the password file is present path={{ cifmw_openshift_login_password_file | default(cifmw_openshift_password_file) }}] *** 2026-01-21 11:09:52.389702 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.045) 0:02:36.029 ***** 2026-01-21 11:09:52.389725 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.045) 0:02:36.027 ***** 2026-01-21 11:09:52.389740 | controller | skipping: [controller] 2026-01-21 11:09:52.419667 | controller | 2026-01-21 11:09:52.419705 | controller | TASK [openshift_login : Fetch user password content src={{ cifmw_openshift_login_password_file | default(cifmw_openshift_password_file) }}] *** 2026-01-21 11:09:52.419736 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.031) 0:02:36.061 ***** 2026-01-21 11:09:52.419748 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.031) 0:02:36.059 ***** 2026-01-21 11:09:52.419760 | controller | skipping: [controller] 2026-01-21 11:09:52.449865 | controller | 2026-01-21 11:09:52.449906 | controller | TASK [openshift_login : Set user password as a fact cifmw_openshift_login_password={{ cifmw_openshift_login_password_file_slurp.content | b64decode }}, cacheable=True] *** 2026-01-21 11:09:52.449915 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.030) 0:02:36.091 ***** 2026-01-21 11:09:52.449923 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.030) 0:02:36.090 ***** 2026-01-21 11:09:52.449935 | controller | skipping: [controller] 2026-01-21 11:09:52.492636 | controller | 2026-01-21 11:09:52.492683 | controller | TASK [openshift_login : Set role variables cifmw_openshift_login_kubeconfig={{ 2026-01-21 11:09:52.492694 | controller | cifmw_openshift_login_kubeconfig | 2026-01-21 11:09:52.492703 | controller | default(cifmw_openshift_kubeconfig) | 2026-01-21 11:09:52.492748 | controller | default( 2026-01-21 11:09:52.492762 | controller | ansible_env.KUBECONFIG if 'KUBECONFIG' in ansible_env else 2026-01-21 11:09:52.492770 | controller | cifmw_openshift_login_kubeconfig_default_path 2026-01-21 11:09:52.492778 | controller | ) | trim 2026-01-21 11:09:52.492795 | controller | }}, cifmw_openshift_login_user={{ cifmw_openshift_login_user | default(cifmw_openshift_user) | default(omit) }}, cifmw_openshift_login_password={{ cifmw_openshift_login_password | default(cifmw_openshift_password) | default(omit) }}, cifmw_openshift_login_api={{ cifmw_openshift_login_api | default(cifmw_openshift_api) | default(omit) }}, cifmw_openshift_login_cert_login={{ cifmw_openshift_login_cert_login | default(false)}}, cifmw_openshift_login_provided_token={{ cifmw_openshift_provided_token | default(omit) }}, cacheable=True] *** 2026-01-21 11:09:52.492816 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.030) 0:02:36.121 ***** 2026-01-21 11:09:52.492824 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.030) 0:02:36.120 ***** 2026-01-21 11:09:52.492837 | controller | ok: [controller] 2026-01-21 11:09:52.731624 | controller | 2026-01-21 11:09:52.731703 | controller | TASK [openshift_login : Check if kubeconfig exists path={{ cifmw_openshift_login_kubeconfig }}] *** 2026-01-21 11:09:52.731741 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.042) 0:02:36.164 ***** 2026-01-21 11:09:52.731755 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.042) 0:02:36.162 ***** 2026-01-21 11:09:52.731770 | controller | ok: [controller] 2026-01-21 11:09:52.768502 | controller | 2026-01-21 11:09:52.768539 | controller | TASK [openshift_login : Assert that enough data is provided to log in to OpenShift that=cifmw_openshift_login_kubeconfig_stat.stat.exists or (cifmw_openshift_login_provided_token is defined and cifmw_openshift_login_provided_token != '') or ( 2026-01-21 11:09:52.768563 | controller | (cifmw_openshift_login_user is defined) and 2026-01-21 11:09:52.768579 | controller | (cifmw_openshift_login_password is defined) and 2026-01-21 11:09:52.768586 | controller | (cifmw_openshift_login_api is defined) 2026-01-21 11:09:52.768597 | controller | ), msg=If an existing kubeconfig is not provided user/pwd or provided/initial token and API URL must be given] *** 2026-01-21 11:09:52.768605 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.238) 0:02:36.402 ***** 2026-01-21 11:09:52.768612 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.238) 0:02:36.401 ***** 2026-01-21 11:09:52.768765 | controller | ok: [controller] => 2026-01-21 11:09:52.797239 | controller | changed: false 2026-01-21 11:09:52.797280 | controller | msg: All assertions passed 2026-01-21 11:09:52.797288 | controller | 2026-01-21 11:09:52.797296 | controller | TASK [openshift_login : Fetch kubeconfig content src={{ cifmw_openshift_login_kubeconfig }}] *** 2026-01-21 11:09:52.797303 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.037) 0:02:36.440 ***** 2026-01-21 11:09:52.797309 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.037) 0:02:36.438 ***** 2026-01-21 11:09:52.797322 | controller | skipping: [controller] 2026-01-21 11:09:52.825511 | controller | 2026-01-21 11:09:52.825553 | controller | TASK [openshift_login : Fetch x509 key based users cifmw_openshift_login_key_based_users={{ 2026-01-21 11:09:52.825561 | controller | ( 2026-01-21 11:09:52.825568 | controller | cifmw_openshift_login_kubeconfig_content_b64.content | 2026-01-21 11:09:52.825575 | controller | b64decode | 2026-01-21 11:09:52.825582 | controller | from_yaml 2026-01-21 11:09:52.825588 | controller | ). users | default([]) | 2026-01-21 11:09:52.825594 | controller | selectattr('user.client-certificate-data', 'defined') | 2026-01-21 11:09:52.825601 | controller | map(attribute="name") | 2026-01-21 11:09:52.825607 | controller | map("split", "/") | 2026-01-21 11:09:52.825614 | controller | map("first") 2026-01-21 11:09:52.825620 | controller | }}, cacheable=True] *** 2026-01-21 11:09:52.825626 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.028) 0:02:36.468 ***** 2026-01-21 11:09:52.825633 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.028) 0:02:36.467 ***** 2026-01-21 11:09:52.825645 | controller | skipping: [controller] 2026-01-21 11:09:52.854059 | controller | 2026-01-21 11:09:52.854105 | controller | TASK [openshift_login : Assign key based user if not provided and available cifmw_openshift_login_user={{ 2026-01-21 11:09:52.854137 | controller | (cifmw_openshift_login_assume_cert_system_user | ternary('system:', '')) + 2026-01-21 11:09:52.854147 | controller | (cifmw_openshift_login_key_based_users | map('replace', 'system:', '') | unique | first) 2026-01-21 11:09:52.854156 | controller | }}, cifmw_openshift_login_cert_login=True, cacheable=True] *** 2026-01-21 11:09:52.854164 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.028) 0:02:36.497 ***** 2026-01-21 11:09:52.854172 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.028) 0:02:36.495 ***** 2026-01-21 11:09:52.854186 | controller | skipping: [controller] 2026-01-21 11:09:52.889079 | controller | 2026-01-21 11:09:52.889120 | controller | TASK [openshift_login : Set the retry count cifmw_openshift_login_retries_cnt={{ 2026-01-21 11:09:52.889129 | controller | 0 if cifmw_openshift_login_retries_cnt is undefined else 2026-01-21 11:09:52.889136 | controller | cifmw_openshift_login_retries_cnt|int + 1 2026-01-21 11:09:52.889143 | controller | }}] *** 2026-01-21 11:09:52.889149 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.028) 0:02:36.525 ***** 2026-01-21 11:09:52.889156 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.028) 0:02:36.524 ***** 2026-01-21 11:09:52.889167 | controller | ok: [controller] 2026-01-21 11:09:52.923377 | controller | 2026-01-21 11:09:52.923414 | controller | TASK [openshift_login : Fetch token _raw_params=try_login.yml] ***************** 2026-01-21 11:09:52.923423 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.034) 0:02:36.560 ***** 2026-01-21 11:09:52.923429 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.034) 0:02:36.559 ***** 2026-01-21 11:09:52.923441 | controller | included: /home/zuul/src/github.com/openstack-k8s-operators/ci-framework/roles/openshift_login/tasks/try_login.yml for controller 2026-01-21 11:09:52.947991 | controller | 2026-01-21 11:09:52.948029 | controller | TASK [openshift_login : Try get OpenShift access token _raw_params=oc whoami -t] *** 2026-01-21 11:09:52.948038 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.034) 0:02:36.595 ***** 2026-01-21 11:09:52.948057 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.034) 0:02:36.593 ***** 2026-01-21 11:09:52.948070 | controller | skipping: [controller] 2026-01-21 11:09:53.854261 | controller | 2026-01-21 11:09:53.854351 | controller | TASK [openshift_login : Fetch OpenShift token output_dir={{ cifmw_openshift_login_basedir }}/artifacts, script=oc login {%- if cifmw_openshift_login_provided_token is not defined %} {%- if cifmw_openshift_login_user is defined %} -u {{ cifmw_openshift_login_user }} {%- endif %} {%- if cifmw_openshift_login_password is defined %} -p {{ cifmw_openshift_login_password }} {%- endif %} {% else %} --token={{ cifmw_openshift_login_provided_token }} {%- endif %} {%- if cifmw_openshift_login_skip_tls_verify|bool %} --insecure-skip-tls-verify=true {%- endif %} {%- if cifmw_openshift_login_api is defined %} {{ cifmw_openshift_login_api }} {%- endif %}] *** 2026-01-21 11:09:53.854366 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.024) 0:02:36.619 ***** 2026-01-21 11:09:53.854375 | controller | Wednesday 21 January 2026 11:09:52 +0000 (0:00:00.024) 0:02:36.618 ***** 2026-01-21 11:09:53.854393 | controller | Follow script's output here: /home/zuul/ci-framework-data/logs/ci_script_002_fetch_openshift.log 2026-01-21 11:09:53.891115 | controller | changed: [controller] 2026-01-21 11:09:53.891192 | controller | 2026-01-21 11:09:53.891201 | controller | TASK [openshift_login : Ensure kubeconfig is provided that=cifmw_openshift_login_kubeconfig != ""] *** 2026-01-21 11:09:53.891209 | controller | Wednesday 21 January 2026 11:09:53 +0000 (0:00:00.905) 0:02:37.525 ***** 2026-01-21 11:09:53.891216 | controller | Wednesday 21 January 2026 11:09:53 +0000 (0:00:00.905) 0:02:37.524 ***** 2026-01-21 11:09:53.891235 | controller | ok: [controller] => 2026-01-21 11:09:54.304837 | controller | changed: false 2026-01-21 11:09:54.304926 | controller | msg: All assertions passed 2026-01-21 11:09:54.304937 | controller | 2026-01-21 11:09:54.304946 | controller | TASK [openshift_login : Fetch new OpenShift access token _raw_params=oc whoami -t] *** 2026-01-21 11:09:54.304955 | controller | Wednesday 21 January 2026 11:09:53 +0000 (0:00:00.037) 0:02:37.562 ***** 2026-01-21 11:09:54.304964 | controller | Wednesday 21 January 2026 11:09:53 +0000 (0:00:00.037) 0:02:37.561 ***** 2026-01-21 11:09:54.304986 | controller | changed: [controller] 2026-01-21 11:09:54.341295 | controller | 2026-01-21 11:09:54.341359 | controller | TASK [openshift_login : Set new OpenShift token cifmw_openshift_login_token={{ 2026-01-21 11:09:54.341369 | controller | (not cifmw_openshift_login_new_token_out.skipped | default(false)) | 2026-01-21 11:09:54.341378 | controller | ternary(cifmw_openshift_login_new_token_out.stdout, cifmw_openshift_login_whoami_out.stdout) 2026-01-21 11:09:54.341387 | controller | }}, cacheable=True] *** 2026-01-21 11:09:54.341396 | controller | Wednesday 21 January 2026 11:09:54 +0000 (0:00:00.413) 0:02:37.976 ***** 2026-01-21 11:09:54.341410 | controller | Wednesday 21 January 2026 11:09:54 +0000 (0:00:00.413) 0:02:37.974 ***** 2026-01-21 11:09:54.341424 | controller | ok: [controller] 2026-01-21 11:09:54.714836 | controller | 2026-01-21 11:09:54.714879 | controller | TASK [openshift_login : Fetch OpenShift API URL _raw_params=oc whoami --show-server=true] *** 2026-01-21 11:09:54.714888 | controller | Wednesday 21 January 2026 11:09:54 +0000 (0:00:00.036) 0:02:38.012 ***** 2026-01-21 11:09:54.714896 | controller | Wednesday 21 January 2026 11:09:54 +0000 (0:00:00.036) 0:02:38.011 ***** 2026-01-21 11:09:54.714907 | controller | changed: [controller] 2026-01-21 11:09:55.113388 | controller | 2026-01-21 11:09:55.113571 | controller | TASK [openshift_login : Fetch OpenShift kubeconfig context _raw_params=oc whoami -c] *** 2026-01-21 11:09:55.113582 | controller | Wednesday 21 January 2026 11:09:54 +0000 (0:00:00.373) 0:02:38.386 ***** 2026-01-21 11:09:55.113589 | controller | Wednesday 21 January 2026 11:09:54 +0000 (0:00:00.373) 0:02:38.384 ***** 2026-01-21 11:09:55.113601 | controller | changed: [controller] 2026-01-21 11:09:55.529258 | controller | 2026-01-21 11:09:55.529299 | controller | TASK [openshift_login : Fetch OpenShift current user _raw_params=oc whoami] **** 2026-01-21 11:09:55.529309 | controller | Wednesday 21 January 2026 11:09:55 +0000 (0:00:00.398) 0:02:38.784 ***** 2026-01-21 11:09:55.529315 | controller | Wednesday 21 January 2026 11:09:55 +0000 (0:00:00.398) 0:02:38.783 ***** 2026-01-21 11:09:55.529328 | controller | changed: [controller] 2026-01-21 11:09:55.572686 | controller | 2026-01-21 11:09:55.572766 | controller | TASK [openshift_login : Set OpenShift user, context and API facts cifmw_openshift_login_api={{ cifmw_openshift_login_api_out.stdout }}, cifmw_openshift_login_context={{ cifmw_openshift_login_context_out.stdout }}, cifmw_openshift_login_user={{ _oauth_user }}, cifmw_openshift_kubeconfig={{ cifmw_openshift_login_kubeconfig }}, cifmw_openshift_api={{ cifmw_openshift_login_api_out.stdout }}, cifmw_openshift_context={{ cifmw_openshift_login_context_out.stdout }}, cifmw_openshift_user={{ _oauth_user }}, cifmw_openshift_token={{ cifmw_openshift_login_token | default(omit) }}, cifmw_install_yamls_environment={{ ( cifmw_install_yamls_environment | 2026-01-21 11:09:55.572779 | controller | combine({'KUBECONFIG': cifmw_openshift_login_kubeconfig}) 2026-01-21 11:09:55.572787 | controller | ) if cifmw_install_yamls_environment is defined else omit 2026-01-21 11:09:55.572793 | controller | }}, cacheable=True] *** 2026-01-21 11:09:55.572800 | controller | Wednesday 21 January 2026 11:09:55 +0000 (0:00:00.416) 0:02:39.200 ***** 2026-01-21 11:09:55.572807 | controller | Wednesday 21 January 2026 11:09:55 +0000 (0:00:00.416) 0:02:39.199 ***** 2026-01-21 11:09:55.572819 | controller | ok: [controller] 2026-01-21 11:09:56.312585 | controller | 2026-01-21 11:09:56.312637 | controller | TASK [openshift_login : Create the openshift_login parameters file dest={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts/parameters/openshift-login-params.yml, content={{ cifmw_openshift_login_params_content | from_yaml | to_nice_yaml }}, mode=0600] *** 2026-01-21 11:09:56.312661 | controller | Wednesday 21 January 2026 11:09:55 +0000 (0:00:00.043) 0:02:39.244 ***** 2026-01-21 11:09:56.312672 | controller | Wednesday 21 January 2026 11:09:55 +0000 (0:00:00.043) 0:02:39.243 ***** 2026-01-21 11:09:56.312688 | controller | changed: [controller] 2026-01-21 11:09:56.713650 | controller | 2026-01-21 11:09:56.713751 | controller | TASK [openshift_login : Read the install yamls parameters file path={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts/parameters/install-yamls-params.yml] *** 2026-01-21 11:09:56.713799 | controller | Wednesday 21 January 2026 11:09:56 +0000 (0:00:00.739) 0:02:39.983 ***** 2026-01-21 11:09:56.713814 | controller | Wednesday 21 January 2026 11:09:56 +0000 (0:00:00.739) 0:02:39.982 ***** 2026-01-21 11:09:56.713833 | controller | ok: [controller] 2026-01-21 11:09:57.561038 | controller | 2026-01-21 11:09:57.561234 | controller | TASK [openshift_login : Append the KUBECONFIG to the install yamls parameters content={{ 2026-01-21 11:09:57.561253 | controller | cifmw_openshift_login_install_yamls_artifacts_slurp['content'] | b64decode | from_yaml | 2026-01-21 11:09:57.561260 | controller | combine( 2026-01-21 11:09:57.561280 | controller | { 2026-01-21 11:09:57.561287 | controller | 'cifmw_install_yamls_environment': { 2026-01-21 11:09:57.561298 | controller | 'KUBECONFIG': cifmw_openshift_login_kubeconfig 2026-01-21 11:09:57.561305 | controller | } 2026-01-21 11:09:57.561311 | controller | }, recursive=true) | to_nice_yaml 2026-01-21 11:09:57.561322 | controller | }}, dest={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts/parameters/install-yamls-params.yml, mode=0600] *** 2026-01-21 11:09:57.561334 | controller | Wednesday 21 January 2026 11:09:56 +0000 (0:00:00.400) 0:02:40.384 ***** 2026-01-21 11:09:57.561344 | controller | Wednesday 21 January 2026 11:09:56 +0000 (0:00:00.400) 0:02:40.383 ***** 2026-01-21 11:09:57.561371 | controller | changed: [controller] 2026-01-21 11:09:57.861441 | controller | 2026-01-21 11:09:57.861492 | controller | TASK [openshift_setup : Ensure output directory exists path={{ cifmw_openshift_setup_basedir }}/artifacts, state=directory, mode=0755] *** 2026-01-21 11:09:57.861504 | controller | Wednesday 21 January 2026 11:09:57 +0000 (0:00:00.847) 0:02:41.231 ***** 2026-01-21 11:09:57.861530 | controller | Wednesday 21 January 2026 11:09:57 +0000 (0:00:00.847) 0:02:41.230 ***** 2026-01-21 11:09:57.861549 | controller | ok: [controller] 2026-01-21 11:09:57.910442 | controller | 2026-01-21 11:09:57.910490 | controller | TASK [openshift_setup : Fetch namespaces to create cifmw_openshift_setup_namespaces={{ 2026-01-21 11:09:57.910501 | controller | (( 2026-01-21 11:09:57.910510 | controller | ([cifmw_install_yamls_defaults['NAMESPACE']] + 2026-01-21 11:09:57.910520 | controller | ([cifmw_install_yamls_defaults['OPERATOR_NAMESPACE']] if 'OPERATOR_NAMESPACE' is in cifmw_install_yamls_defaults else []) 2026-01-21 11:09:57.910529 | controller | ) if cifmw_install_yamls_defaults is defined else [] 2026-01-21 11:09:57.910537 | controller | ) + cifmw_openshift_setup_create_namespaces) | unique 2026-01-21 11:09:57.910546 | controller | }}] *** 2026-01-21 11:09:57.910555 | controller | Wednesday 21 January 2026 11:09:57 +0000 (0:00:00.300) 0:02:41.532 ***** 2026-01-21 11:09:57.910563 | controller | Wednesday 21 January 2026 11:09:57 +0000 (0:00:00.300) 0:02:41.531 ***** 2026-01-21 11:09:57.910579 | controller | ok: [controller] 2026-01-21 11:10:00.318584 | controller | 2026-01-21 11:10:00.318637 | controller | TASK [openshift_setup : Create required namespaces kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit) }}, name={{ item }}, kind=Namespace, state=present] *** 2026-01-21 11:10:00.318652 | controller | Wednesday 21 January 2026 11:09:57 +0000 (0:00:00.048) 0:02:41.581 ***** 2026-01-21 11:10:00.318660 | controller | Wednesday 21 January 2026 11:09:57 +0000 (0:00:00.048) 0:02:41.580 ***** 2026-01-21 11:10:00.318671 | controller | changed: [controller] => (item=openstack) 2026-01-21 11:10:00.343937 | controller | changed: [controller] => (item=openstack-operators) 2026-01-21 11:10:00.343985 | controller | 2026-01-21 11:10:00.343995 | controller | TASK [openshift_setup : Get internal OpenShift registry route kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, kind=Route, name=default-route, namespace=openshift-image-registry] *** 2026-01-21 11:10:00.344020 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:02.408) 0:02:43.990 ***** 2026-01-21 11:10:00.344029 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:02.408) 0:02:43.988 ***** 2026-01-21 11:10:00.344045 | controller | skipping: [controller] 2026-01-21 11:10:00.383196 | controller | 2026-01-21 11:10:00.383261 | controller | TASK [openshift_setup : Allow anonymous image-pulls in CRC registry for targeted namespaces state=present, kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, definition={'kind': 'RoleBinding', 'apiVersion': 'rbac.authorization.k8s.io/v1', 'metadata': {'name': 'system:image-puller', 'namespace': '{{ item }}'}, 'subjects': [{'kind': 'User', 'name': 'system:anonymous'}, {'kind': 'User', 'name': 'system:unauthenticated'}], 'roleRef': {'kind': 'ClusterRole', 'name': 'system:image-puller'}}] *** 2026-01-21 11:10:00.383274 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.025) 0:02:44.015 ***** 2026-01-21 11:10:00.383290 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.025) 0:02:44.014 ***** 2026-01-21 11:10:00.383305 | controller | skipping: [controller] => (item=openstack) 2026-01-21 11:10:00.417868 | controller | skipping: [controller] => (item=openstack-operators) 2026-01-21 11:10:00.417915 | controller | skipping: [controller] 2026-01-21 11:10:00.417925 | controller | 2026-01-21 11:10:00.417934 | controller | TASK [openshift_setup : Wait for the image registry to be ready kind=Deployment, name=image-registry, namespace=openshift-image-registry, kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, wait=True, wait_sleep=10, wait_timeout=600, wait_condition={'type': 'Available', 'status': 'True'}] *** 2026-01-21 11:10:00.417954 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.039) 0:02:44.054 ***** 2026-01-21 11:10:00.417963 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.039) 0:02:44.053 ***** 2026-01-21 11:10:00.417977 | controller | skipping: [controller] 2026-01-21 11:10:00.452943 | controller | 2026-01-21 11:10:00.452989 | controller | TASK [openshift_setup : Login into OpenShift internal registry output_dir={{ cifmw_openshift_setup_basedir }}/artifacts, script=podman login -u {{ cifmw_openshift_user }} -p {{ cifmw_openshift_token }} {%- if cifmw_openshift_setup_skip_internal_registry_tls_verify|bool %} --tls-verify=false {%- endif %} {{ cifmw_openshift_setup_registry_default_route.resources[0].spec.host }}] *** 2026-01-21 11:10:00.452999 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.034) 0:02:44.089 ***** 2026-01-21 11:10:00.453007 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.034) 0:02:44.088 ***** 2026-01-21 11:10:00.453021 | controller | skipping: [controller] 2026-01-21 11:10:00.484047 | controller | 2026-01-21 11:10:00.484093 | controller | TASK [Ensure we have custom CA installed on host role=install_ca] ************** 2026-01-21 11:10:00.484103 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.035) 0:02:44.124 ***** 2026-01-21 11:10:00.484111 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.035) 0:02:44.123 ***** 2026-01-21 11:10:00.484124 | controller | skipping: [controller] 2026-01-21 11:10:00.512482 | controller | 2026-01-21 11:10:00.512527 | controller | TASK [openshift_setup : Update ca bundle _raw_params=update-ca-trust extract] *** 2026-01-21 11:10:00.512537 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.031) 0:02:44.155 ***** 2026-01-21 11:10:00.512544 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.031) 0:02:44.154 ***** 2026-01-21 11:10:00.512558 | controller | skipping: [controller] 2026-01-21 11:10:00.540973 | controller | 2026-01-21 11:10:00.541016 | controller | TASK [openshift_setup : Slurp CAs file src={{ cifmw_openshift_setup_ca_bundle_path }}] *** 2026-01-21 11:10:00.541025 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.028) 0:02:44.184 ***** 2026-01-21 11:10:00.541032 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.028) 0:02:44.182 ***** 2026-01-21 11:10:00.541043 | controller | skipping: [controller] 2026-01-21 11:10:00.570039 | controller | 2026-01-21 11:10:00.570160 | controller | TASK [openshift_setup : Create config map with registry CAs kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, definition={'apiVersion': 'v1', 'kind': 'ConfigMap', 'metadata': {'namespace': 'openshift-config', 'name': 'registry-cas'}, 'data': '{{ _config_map_data | items2dict }}'}] *** 2026-01-21 11:10:00.570176 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.028) 0:02:44.212 ***** 2026-01-21 11:10:00.570185 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.028) 0:02:44.211 ***** 2026-01-21 11:10:00.570197 | controller | skipping: [controller] 2026-01-21 11:10:00.601686 | controller | 2026-01-21 11:10:00.601773 | controller | TASK [openshift_setup : Install Red Hat CA for pulling images from internal registry kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, merge_type=merge, definition={'apiVersion': 'config.openshift.io/v1', 'kind': 'Image', 'metadata': {'name': 'cluster'}, 'spec': {'additionalTrustedCA': {'name': 'registry-cas'}}}] *** 2026-01-21 11:10:00.601820 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.029) 0:02:44.241 ***** 2026-01-21 11:10:00.601840 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.029) 0:02:44.240 ***** 2026-01-21 11:10:00.601858 | controller | skipping: [controller] 2026-01-21 11:10:00.632510 | controller | 2026-01-21 11:10:00.632569 | controller | TASK [openshift_setup : Add insecure registry kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, merge_type=merge, definition={'apiVersion': 'config.openshift.io/v1', 'kind': 'Image', 'metadata': {'name': 'cluster'}, 'spec': {'registrySources': {'insecureRegistries': ['{{ cifmw_update_containers_registry }}'], 'allowedRegistries': '{{ all_registries }}'}}}] *** 2026-01-21 11:10:00.632580 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.031) 0:02:44.273 ***** 2026-01-21 11:10:00.632587 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.031) 0:02:44.271 ***** 2026-01-21 11:10:00.632599 | controller | skipping: [controller] 2026-01-21 11:10:00.666071 | controller | 2026-01-21 11:10:00.666112 | controller | TASK [openshift_setup : Create a ICSP with repository digest mirrors kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, definition={'apiVersion': 'operator.openshift.io/v1alpha1', 'kind': 'ImageContentSourcePolicy', 'metadata': {'name': 'registry-digest-mirrors'}, 'spec': {'repositoryDigestMirrors': '{{ cifmw_openshift_setup_digest_mirrors }}'}}] *** 2026-01-21 11:10:00.666123 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.031) 0:02:44.304 ***** 2026-01-21 11:10:00.666131 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.030) 0:02:44.302 ***** 2026-01-21 11:10:00.666166 | controller | skipping: [controller] 2026-01-21 11:10:01.820204 | controller | 2026-01-21 11:10:01.820251 | controller | TASK [openshift_setup : Gather network.operator info kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, api_version=operator.openshift.io/v1, kind=Network, name=cluster] *** 2026-01-21 11:10:01.820272 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.033) 0:02:44.337 ***** 2026-01-21 11:10:01.820283 | controller | Wednesday 21 January 2026 11:10:00 +0000 (0:00:00.033) 0:02:44.336 ***** 2026-01-21 11:10:01.820299 | controller | ok: [controller] 2026-01-21 11:10:03.079423 | controller | 2026-01-21 11:10:03.079574 | controller | TASK [openshift_setup : Patch network operator api_version=operator.openshift.io/v1, kubeconfig={{ cifmw_openshift_kubeconfig }}, kind=Network, name=cluster, persist_config=True, patch=[{'path': '/spec/defaultNetwork/ovnKubernetesConfig/gatewayConfig/routingViaHost', 'value': True, 'op': 'replace'}, {'path': '/spec/defaultNetwork/ovnKubernetesConfig/gatewayConfig/ipForwarding', 'value': 'Global', 'op': 'replace'}]] *** 2026-01-21 11:10:03.079599 | controller | Wednesday 21 January 2026 11:10:01 +0000 (0:00:01.153) 0:02:45.491 ***** 2026-01-21 11:10:03.079614 | controller | Wednesday 21 January 2026 11:10:01 +0000 (0:00:01.153) 0:02:45.489 ***** 2026-01-21 11:10:03.079634 | controller | changed: [controller] 2026-01-21 11:10:04.125031 | controller | 2026-01-21 11:10:04.125077 | controller | TASK [openshift_setup : Patch samples registry configuration kubeconfig={{ cifmw_openshift_kubeconfig }}, api_key={{ cifmw_openshift_token | default(omit)}}, context={{ cifmw_openshift_context | default(omit)}}, api_version=samples.operator.openshift.io/v1, kind=Config, name=cluster, patch=[{'op': 'replace', 'path': '/spec/samplesRegistry', 'value': 'registry.redhat.io'}]] *** 2026-01-21 11:10:04.125088 | controller | Wednesday 21 January 2026 11:10:03 +0000 (0:00:01.259) 0:02:46.750 ***** 2026-01-21 11:10:04.125095 | controller | Wednesday 21 January 2026 11:10:03 +0000 (0:00:01.259) 0:02:46.748 ***** 2026-01-21 11:10:04.125106 | controller | changed: [controller] 2026-01-21 11:10:04.150776 | controller | 2026-01-21 11:10:04.150815 | controller | TASK [openshift_setup : Delete the pods from openshift-marketplace namespace kind=Pod, state=absent, delete_all=True, kubeconfig={{ cifmw_openshift_kubeconfig }}, namespace=openshift-marketplace] *** 2026-01-21 11:10:04.150835 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:01.046) 0:02:47.796 ***** 2026-01-21 11:10:04.150843 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:01.046) 0:02:47.795 ***** 2026-01-21 11:10:04.150855 | controller | skipping: [controller] 2026-01-21 11:10:04.181870 | controller | 2026-01-21 11:10:04.181928 | controller | TASK [openshift_setup : Wait for openshift-marketplace pods to be running _raw_params=oc wait pod --all --for=condition=Ready -n openshift-marketplace --timeout=1m] *** 2026-01-21 11:10:04.181941 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:00.025) 0:02:47.822 ***** 2026-01-21 11:10:04.181948 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:00.025) 0:02:47.820 ***** 2026-01-21 11:10:04.181959 | controller | skipping: [controller] 2026-01-21 11:10:04.276944 | controller | 2026-01-21 11:10:04.276997 | controller | TASK [Deploy Observability operator. name=openshift_obs] *********************** 2026-01-21 11:10:04.277008 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:00.031) 0:02:47.853 ***** 2026-01-21 11:10:04.277016 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:00.031) 0:02:47.852 ***** 2026-01-21 11:10:04.277057 | controller | 2026-01-21 11:10:05.682610 | controller | TASK [openshift_obs : Install cluster observability operator. definition={{cifmw_openshift_obs_definition }}, kubeconfig={{ cifmw_openshift_kubeconfig }}, state=present] *** 2026-01-21 11:10:05.682664 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:00.094) 0:02:47.948 ***** 2026-01-21 11:10:05.682675 | controller | Wednesday 21 January 2026 11:10:04 +0000 (0:00:00.094) 0:02:47.946 ***** 2026-01-21 11:10:05.682690 | controller | changed: [controller] 2026-01-21 11:11:17.177395 | controller | 2026-01-21 11:11:17.177649 | controller | TASK [openshift_obs : Wait for observability operator deployment kind=Deployment, namespace=openshift-operators, name=observability-operator, wait=True, wait_timeout=300, wait_condition={'type': 'Available', 'status': 'True'}, kubeconfig={{ cifmw_openshift_kubeconfig }}] *** 2026-01-21 11:11:17.177682 | controller | Wednesday 21 January 2026 11:10:05 +0000 (0:00:01.405) 0:02:49.354 ***** 2026-01-21 11:11:17.177702 | controller | Wednesday 21 January 2026 11:10:05 +0000 (0:00:01.405) 0:02:49.352 ***** 2026-01-21 11:11:17.177772 | controller | ok: [controller] 2026-01-21 11:11:18.374753 | controller | 2026-01-21 11:11:18.374943 | controller | TASK [openshift_obs : Wait for observability-operator pod kind=Pod, namespace=openshift-operators, label_selectors=['app.kubernetes.io/name = observability-operator'], wait=True, wait_timeout=300, wait_condition={'type': 'Ready', 'status': 'True'}, kubeconfig={{ cifmw_openshift_kubeconfig }}] *** 2026-01-21 11:11:18.374971 | controller | Wednesday 21 January 2026 11:11:17 +0000 (0:01:11.492) 0:04:00.846 ***** 2026-01-21 11:11:18.375010 | controller | Wednesday 21 January 2026 11:11:17 +0000 (0:01:11.492) 0:04:00.845 ***** 2026-01-21 11:11:18.375042 | controller | ok: [controller] 2026-01-21 11:11:18.413481 | controller | 2026-01-21 11:11:18.413530 | controller | TASK [Deploy Metal3 BMHs name=deploy_bmh] ************************************** 2026-01-21 11:11:18.413541 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:01.198) 0:04:02.044 ***** 2026-01-21 11:11:18.413550 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:01.198) 0:04:02.043 ***** 2026-01-21 11:11:18.413565 | controller | skipping: [controller] 2026-01-21 11:11:18.446320 | controller | 2026-01-21 11:11:18.446365 | controller | TASK [Install certmanager operator role name=cert_manager] ********************* 2026-01-21 11:11:18.446377 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.039) 0:04:02.084 ***** 2026-01-21 11:11:18.446385 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.039) 0:04:02.083 ***** 2026-01-21 11:11:18.446401 | controller | skipping: [controller] 2026-01-21 11:11:18.480403 | controller | 2026-01-21 11:11:18.480450 | controller | TASK [Configure hosts networking using nmstate name=ci_nmstate] **************** 2026-01-21 11:11:18.480466 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.032) 0:04:02.117 ***** 2026-01-21 11:11:18.480472 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.032) 0:04:02.116 ***** 2026-01-21 11:11:18.480485 | controller | skipping: [controller] 2026-01-21 11:11:18.511904 | controller | 2026-01-21 11:11:18.511945 | controller | TASK [Configure multus networks name=ci_multus] ******************************** 2026-01-21 11:11:18.511954 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.034) 0:04:02.151 ***** 2026-01-21 11:11:18.511960 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.034) 0:04:02.150 ***** 2026-01-21 11:11:18.511984 | controller | skipping: [controller] 2026-01-21 11:11:18.544613 | controller | 2026-01-21 11:11:18.544659 | controller | TASK [Deploy Sushy Emulator service pod name=sushy_emulator] ******************* 2026-01-21 11:11:18.544668 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.031) 0:04:02.183 ***** 2026-01-21 11:11:18.544675 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.031) 0:04:02.182 ***** 2026-01-21 11:11:18.544687 | controller | skipping: [controller] 2026-01-21 11:11:18.578378 | controller | 2026-01-21 11:11:18.578427 | controller | TASK [Setup Libvirt on controller name=libvirt_manager] ************************ 2026-01-21 11:11:18.578438 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.032) 0:04:02.215 ***** 2026-01-21 11:11:18.578447 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.032) 0:04:02.214 ***** 2026-01-21 11:11:18.578462 | controller | skipping: [controller] 2026-01-21 11:11:18.614202 | controller | 2026-01-21 11:11:18.614242 | controller | TASK [Prepare container package builder name=pkg_build] ************************ 2026-01-21 11:11:18.614251 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.033) 0:04:02.249 ***** 2026-01-21 11:11:18.614258 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.033) 0:04:02.248 ***** 2026-01-21 11:11:18.614270 | controller | skipping: [controller] 2026-01-21 11:11:18.694745 | controller | 2026-01-21 11:11:18.694795 | controller | TASK [run_hook : Assert parameters are valid quiet=True, that=['_list_hooks is not string', '_list_hooks is not mapping', '_list_hooks is iterable', '(hooks | default([])) is not string', '(hooks | default([])) is not mapping', '(hooks | default([])) is iterable']] *** 2026-01-21 11:11:18.694807 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.036) 0:04:02.285 ***** 2026-01-21 11:11:18.694816 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.035) 0:04:02.284 ***** 2026-01-21 11:11:18.694831 | controller | ok: [controller] 2026-01-21 11:11:18.894132 | controller | 2026-01-21 11:11:18.894201 | controller | TASK [run_hook : Assert single hooks are all mappings quiet=True, that=['_not_mapping_hooks | length == 0'], msg=All single hooks must be a list of mappings or a mapping.] *** 2026-01-21 11:11:18.894219 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.080) 0:04:02.366 ***** 2026-01-21 11:11:18.894229 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.080) 0:04:02.364 ***** 2026-01-21 11:11:18.894244 | controller | ok: [controller] 2026-01-21 11:11:19.124771 | controller | 2026-01-21 11:11:19.124814 | controller | TASK [run_hook : Loop on hooks for post_infra _raw_params={{ hook.type }}.yml] *** 2026-01-21 11:11:19.124833 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.199) 0:04:02.565 ***** 2026-01-21 11:11:19.124840 | controller | Wednesday 21 January 2026 11:11:18 +0000 (0:00:00.199) 0:04:02.563 ***** 2026-01-21 11:11:19.124852 | controller | skipping: [controller] 2026-01-21 11:11:19.209817 | controller | 2026-01-21 11:11:19.209865 | controller | TASK [Load parameters dir={{ item }}] ****************************************** 2026-01-21 11:11:19.209876 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.231) 0:04:02.796 ***** 2026-01-21 11:11:19.209885 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.230) 0:04:02.794 ***** 2026-01-21 11:11:19.209901 | controller | ok: [controller] => (item=/home/zuul/ci-framework-data/artifacts/parameters) 2026-01-21 11:11:19.255500 | controller | ok: [controller] => (item=/etc/ci/env) 2026-01-21 11:11:19.255549 | controller | 2026-01-21 11:11:19.255559 | controller | TASK [Ensure that the isolated net was configured for crc that=['crc_ci_bootstrap_networks_out is defined', 'crc_ci_bootstrap_networks_out[_crc_hostname] is defined', "crc_ci_bootstrap_networks_out[_crc_hostname]['default'] is defined"]] *** 2026-01-21 11:11:19.255592 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.084) 0:04:02.881 ***** 2026-01-21 11:11:19.255601 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.084) 0:04:02.879 ***** 2026-01-21 11:11:19.255617 | controller | ok: [controller] => 2026-01-21 11:11:19.347780 | controller | changed: false 2026-01-21 11:11:19.347825 | controller | msg: All assertions passed 2026-01-21 11:11:19.347835 | controller | 2026-01-21 11:11:19.347856 | controller | TASK [Set facts for further usage within the framework cifmw_edpm_prepare_extra_vars={'NNCP_INTERFACE': '{{ crc_ci_bootstrap_networks_out.crc.default.iface }}', 'NETWORK_MTU': '{{ crc_ci_bootstrap_networks_out.crc.default.mtu }}', 'NNCP_DNS_SERVER': "{{\n cifmw_nncp_dns_server |\n default(crc_ci_bootstrap_networks_out[_crc_hostname].default.ip) |\n split('/') | first\n}}"}] *** 2026-01-21 11:11:19.347866 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.045) 0:04:02.926 ***** 2026-01-21 11:11:19.347874 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.045) 0:04:02.925 ***** 2026-01-21 11:11:19.347889 | controller | ok: [controller] 2026-01-21 11:11:19.414328 | controller | 2026-01-21 11:11:19.414369 | controller | PLAY [Deploy Openstack Operators] ********************************************** 2026-01-21 11:11:19.414377 | controller | 2026-01-21 11:11:19.414384 | controller | TASK [Use the locally built operators if any _local_operators_indexes={{ 2026-01-21 11:11:19.414390 | controller | _local_operators_indexes|default({}) | 2026-01-21 11:11:19.414397 | controller | combine({ item.key.split('-')[0]|upper+'_IMG': 2026-01-21 11:11:19.414403 | controller | cifmw_operator_build_output['operators'][item.key].image_catalog}) 2026-01-21 11:11:19.414410 | controller | }}] *** 2026-01-21 11:11:19.414417 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.092) 0:04:03.019 ***** 2026-01-21 11:11:19.414423 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.092) 0:04:03.017 ***** 2026-01-21 11:11:19.414445 | controller | ok: [controller] => (item={'key': 'openstack-operator', 'value': {'git_commit_hash': '38e630804dada625f7b015f13f3ac5bb7192f4dd', 'git_src_dir': '/home/zuul-worker/src/github.com/openstack-k8s-operators/openstack-operator', 'image': '38.102.83.182:5001/openstack-k8s-operators/openstack-operator:38e630804dada625f7b015f13f3ac5bb7192f4dd', 'image_bundle': '38.102.83.182:5001/openstack-k8s-operators/openstack-operator-bundle:38e630804dada625f7b015f13f3ac5bb7192f4dd', 'image_catalog': '38.102.83.182:5001/openstack-k8s-operators/openstack-operator-index:38e630804dada625f7b015f13f3ac5bb7192f4dd'}}) 2026-01-21 11:11:19.454151 | controller | ok: [controller] => (item={'key': 'watcher-operator', 'value': {'git_commit_hash': '111af6c57cd1be8032328ced6eb7a058a201c4b1', 'git_src_dir': '/home/zuul-worker/src/github.com/openstack-k8s-operators/watcher-operator', 'image': '38.102.83.182:5001/openstack-k8s-operators/watcher-operator:111af6c57cd1be8032328ced6eb7a058a201c4b1', 'image_bundle': '38.102.83.182:5001/openstack-k8s-operators/watcher-operator-bundle:111af6c57cd1be8032328ced6eb7a058a201c4b1', 'image_catalog': '38.102.83.182:5001/openstack-k8s-operators/watcher-operator-index:111af6c57cd1be8032328ced6eb7a058a201c4b1'}}) 2026-01-21 11:11:19.454199 | controller | 2026-01-21 11:11:19.454211 | controller | TASK [Set install_yamls Makefile environment variables cifmw_edpm_prepare_common_env={{ 2026-01-21 11:11:19.454221 | controller | cifmw_install_yamls_environment | 2026-01-21 11:11:19.454230 | controller | combine({'PATH': cifmw_path}) | 2026-01-21 11:11:19.454240 | controller | combine(cifmw_edpm_prepare_extra_vars | default({})) 2026-01-21 11:11:19.454249 | controller | }}, cifmw_edpm_prepare_operators_build_output={{ operators_build_output }}, cifmw_edpm_prepare_make_openstack_env={{ _local_operators_indexes | combine(_openstack_operator_images) }}] *** 2026-01-21 11:11:19.454258 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.066) 0:04:03.085 ***** 2026-01-21 11:11:19.454267 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.066) 0:04:03.084 ***** 2026-01-21 11:11:19.454283 | controller | ok: [controller] 2026-01-21 11:11:19.944843 | controller | 2026-01-21 11:11:19.944893 | controller | TASK [detect if openstack operator is installed _raw_params=oc get sub --ignore-not-found=true -n openstack-operators -o name openstack-operator] *** 2026-01-21 11:11:19.944902 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.039) 0:04:03.125 ***** 2026-01-21 11:11:19.944909 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.039) 0:04:03.124 ***** 2026-01-21 11:11:19.944922 | controller | changed: [controller] 2026-01-21 11:11:19.981369 | controller | 2026-01-21 11:11:19.981416 | controller | TASK [Install openstack operator and wait for the csv to succeed name=install_yamls_makes, tasks_from=make_openstack_init] *** 2026-01-21 11:11:19.981426 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.490) 0:04:03.616 ***** 2026-01-21 11:11:19.981435 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.490) 0:04:03.614 ***** 2026-01-21 11:11:19.981450 | controller | 2026-01-21 11:11:20.023792 | controller | TASK [install_yamls_makes : Debug make_openstack_init_env var=make_openstack_init_env] *** 2026-01-21 11:11:20.023833 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.036) 0:04:03.652 ***** 2026-01-21 11:11:20.023841 | controller | Wednesday 21 January 2026 11:11:19 +0000 (0:00:00.036) 0:04:03.651 ***** 2026-01-21 11:11:20.023854 | controller | ok: [controller] => 2026-01-21 11:11:20.056218 | controller | make_openstack_init_env: 2026-01-21 11:11:20.056263 | controller | BMO_SETUP: false 2026-01-21 11:11:20.056272 | controller | CHECKOUT_FROM_OPENSTACK_REF: 'true' 2026-01-21 11:11:20.056280 | controller | KUBECONFIG: /home/zuul/.crc/machines/crc/kubeconfig 2026-01-21 11:11:20.056288 | controller | NETWORK_MTU: 1500 2026-01-21 11:11:20.056295 | controller | NNCP_DNS_SERVER: 192.168.122.10 2026-01-21 11:11:20.056303 | controller | NNCP_INTERFACE: ens7 2026-01-21 11:11:20.056346 | controller | OPENSTACK_BUNDLE_IMG: 38.102.83.182:5001/openstack-k8s-operators/openstack-operator-bundle:38e630804dada625f7b015f13f3ac5bb7192f4dd 2026-01-21 11:11:20.056365 | controller | OPENSTACK_IMG: 38.102.83.182:5001/openstack-k8s-operators/openstack-operator-index:38e630804dada625f7b015f13f3ac5bb7192f4dd 2026-01-21 11:11:20.056373 | controller | OPENSTACK_K8S_BRANCH: main 2026-01-21 11:11:20.056380 | controller | OUT: /home/zuul/ci-framework-data/artifacts/manifests 2026-01-21 11:11:20.056388 | controller | OUTPUT_DIR: /home/zuul/ci-framework-data/artifacts/edpm 2026-01-21 11:11:20.056395 | controller | PATH: /home/zuul/.crc/bin:/home/zuul/.crc/bin/oc:/home/zuul/bin:/home/zuul/.local/bin:/home/zuul/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin 2026-01-21 11:11:20.056403 | controller | WATCHER_BRANCH: '' 2026-01-21 11:11:20.056411 | controller | WATCHER_IMG: 38.102.83.182:5001/openstack-k8s-operators/watcher-operator-index:111af6c57cd1be8032328ced6eb7a058a201c4b1 2026-01-21 11:11:20.056419 | controller | WATCHER_REPO: /home/zuul/src/github.com/openstack-k8s-operators/watcher-operator 2026-01-21 11:11:20.056426 | controller | 2026-01-21 11:11:20.056434 | controller | TASK [install_yamls_makes : Debug make_openstack_init_params var=make_openstack_init_params] *** 2026-01-21 11:11:20.056442 | controller | Wednesday 21 January 2026 11:11:20 +0000 (0:00:00.042) 0:04:03.695 ***** 2026-01-21 11:11:20.056449 | controller | Wednesday 21 January 2026 11:11:20 +0000 (0:00:00.042) 0:04:03.693 ***** 2026-01-21 11:11:20.056464 | controller | skipping: [controller] 2026-01-21 11:17:06.754398 | controller | 2026-01-21 11:17:06.754576 | controller | TASK [install_yamls_makes : Run openstack_init output_dir={{ cifmw_basedir|default(ansible_user_dir ~ '/ci-framework-data') }}/artifacts, chdir=/home/zuul/src/github.com/openstack-k8s-operators/install_yamls, script=make openstack_init, dry_run={{ make_openstack_init_dryrun|default(false)|bool }}, extra_args={{ dict((make_openstack_init_env|default({})), **(make_openstack_init_params|default({}))) }}] *** 2026-01-21 11:17:06.754595 | controller | Wednesday 21 January 2026 11:11:20 +0000 (0:00:00.032) 0:04:03.727 ***** 2026-01-21 11:17:06.754610 | controller | Wednesday 21 January 2026 11:11:20 +0000 (0:00:00.032) 0:04:03.726 ***** 2026-01-21 11:17:06.754629 | controller | [WARNING]: conditional statements should not include jinja2 templating 2026-01-21 11:17:06.779021 | controller | delimiters such as {{ }} or {% %}. Found: {{ make_openstack_init_until | 2026-01-21 11:17:06.779100 | controller | default(true) }} 2026-01-21 11:17:06.779123 | controller | Follow script's output here: /home/zuul/ci-framework-data/logs/ci_script_003_run_openstack.log 2026-01-21 11:17:06.828938 | controller | changed: [controller] 2026-01-21 11:17:06.828985 | controller | 2026-01-21 11:17:06.828994 | controller | TASK [Run hooks after installing openstack name=run_hook] ********************** 2026-01-21 11:17:06.829002 | controller | Wednesday 21 January 2026 11:17:06 +0000 (0:05:46.722) 0:09:50.450 ***** 2026-01-21 11:17:06.829009 | controller | Wednesday 21 January 2026 11:17:06 +0000 (0:05:46.722) 0:09:50.448 ***** 2026-01-21 11:17:06.829022 | controller | 2026-01-21 11:17:06.945282 | controller | TASK [run_hook : Assert parameters are valid quiet=True, that=['_list_hooks is not string', '_list_hooks is not mapping', '_list_hooks is iterable', '(hooks | default([])) is not string', '(hooks | default([])) is not mapping', '(hooks | default([])) is iterable']] *** 2026-01-21 11:17:06.945338 | controller | Wednesday 21 January 2026 11:17:06 +0000 (0:00:00.050) 0:09:50.500 ***** 2026-01-21 11:17:06.945349 | controller | Wednesday 21 January 2026 11:17:06 +0000 (0:00:00.050) 0:09:50.498 ***** 2026-01-21 11:17:06.945364 | controller | ok: [controller] 2026-01-21 11:17:07.124332 | controller | 2026-01-21 11:17:07.124385 | controller | TASK [run_hook : Assert single hooks are all mappings quiet=True, that=['_not_mapping_hooks | length == 0'], msg=All single hooks must be a list of mappings or a mapping.] *** 2026-01-21 11:17:07.124400 | controller | Wednesday 21 January 2026 11:17:06 +0000 (0:00:00.116) 0:09:50.616 ***** 2026-01-21 11:17:07.124408 | controller | Wednesday 21 January 2026 11:17:06 +0000 (0:00:00.116) 0:09:50.615 ***** 2026-01-21 11:17:07.124420 | controller | ok: [controller] 2026-01-21 11:17:07.278035 | controller | 2026-01-21 11:17:07.278082 | controller | TASK [run_hook : Loop on hooks for post_install_operators_kuttl_from_operator _raw_params={{ hook.type }}.yml] *** 2026-01-21 11:17:07.278094 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.179) 0:09:50.795 ***** 2026-01-21 11:17:07.278103 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.179) 0:09:50.794 ***** 2026-01-21 11:17:07.278119 | controller | included: /home/zuul/src/github.com/openstack-k8s-operators/ci-framework/roles/run_hook/tasks/playbook.yml for controller => (item={'name': 'Deploy watcher operator', 'type': 'playbook', 'source': '/home/zuul/src/github.com/openstack-k8s-operators/watcher-operator/ci/playbooks/deploy_watcher_operator.yaml', 'extra_vars': {'content_provider_os_registry_url': '38.102.83.182:5001/podified-master-centos10', 'watcher_catalog_image': '38.102.83.182:5001/openstack-k8s-operators/watcher-operator-index:111af6c57cd1be8032328ced6eb7a058a201c4b1', 'watcher_services_tag': 'watcher_latest', 'watcher_repo': '/home/zuul/src/github.com/openstack-k8s-operators/watcher-operator'}}) 2026-01-21 11:17:07.341039 | controller | 2026-01-21 11:17:07.341095 | controller | TASK [run_hook : Set playbook path for Deploy watcher operator cifmw_basedir={{ _bdir }}, hook_name={{ _hook_name }}, playbook_path={{ _play | realpath }}, log_path={{ _bdir }}/logs/{{ step }}_{{ _hook_name }}.log, extra_vars=-e namespace={{ cifmw_openstack_namespace }} {%- if hook.extra_vars is defined and hook.extra_vars|length > 0 -%} {% for key,value in hook.extra_vars.items() -%} {%- if key == 'file' %} -e "@{{ value }}" {%- else %} -e "{{ key }}={{ value }}" {%- endif %} {%- endfor %} {%- endif %}] *** 2026-01-21 11:17:07.341106 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.153) 0:09:50.949 ***** 2026-01-21 11:17:07.341114 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.153) 0:09:50.947 ***** 2026-01-21 11:17:07.341126 | controller | ok: [controller] 2026-01-21 11:17:07.616469 | controller | 2026-01-21 11:17:07.616533 | controller | TASK [run_hook : Get file stat path={{ playbook_path }}] *********************** 2026-01-21 11:17:07.616544 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.063) 0:09:51.012 ***** 2026-01-21 11:17:07.616553 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.063) 0:09:51.011 ***** 2026-01-21 11:17:07.616569 | controller | ok: [controller] 2026-01-21 11:17:07.645927 | controller | 2026-01-21 11:17:07.645976 | controller | TASK [run_hook : Fail if playbook doesn't exist msg=Playbook {{ playbook_path }} doesn't seem to exist.] *** 2026-01-21 11:17:07.646002 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.274) 0:09:51.287 ***** 2026-01-21 11:17:07.646012 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.275) 0:09:51.286 ***** 2026-01-21 11:17:07.646028 | controller | skipping: [controller] 2026-01-21 11:17:07.913281 | controller | 2026-01-21 11:17:07.913320 | controller | TASK [run_hook : Get parameters files paths={{ 2026-01-21 11:17:07.913330 | controller | (cifmw_basedir, 'artifacts/parameters') | path_join 2026-01-21 11:17:07.913337 | controller | }}, file_type=file, patterns=*.yml] *** 2026-01-21 11:17:07.913348 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.029) 0:09:51.317 ***** 2026-01-21 11:17:07.913355 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.029) 0:09:51.315 ***** 2026-01-21 11:17:07.913367 | controller | ok: [controller] 2026-01-21 11:17:07.949840 | controller | 2026-01-21 11:17:07.949891 | controller | TASK [run_hook : Add parameters artifacts as extra variables extra_vars={{ extra_vars }} {% for file in cifmw_run_hook_parameters_files.files %} -e "@{{ file.path }}" {%- endfor %}] *** 2026-01-21 11:17:07.949902 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.267) 0:09:51.584 ***** 2026-01-21 11:17:07.949912 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.267) 0:09:51.583 ***** 2026-01-21 11:17:07.949928 | controller | ok: [controller] 2026-01-21 11:17:08.207166 | controller | 2026-01-21 11:17:08.207213 | controller | TASK [run_hook : Ensure log directory exists path={{ log_path | dirname }}, state=directory, mode=0755] *** 2026-01-21 11:17:08.207222 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.036) 0:09:51.621 ***** 2026-01-21 11:17:08.207238 | controller | Wednesday 21 January 2026 11:17:07 +0000 (0:00:00.036) 0:09:51.619 ***** 2026-01-21 11:17:08.207250 | controller | ok: [controller] 2026-01-21 11:17:08.477708 | controller | 2026-01-21 11:17:08.477778 | controller | TASK [run_hook : Ensure artifacts directory exists path={{ cifmw_basedir }}/artifacts, state=directory, mode=0755] *** 2026-01-21 11:17:08.477789 | controller | Wednesday 21 January 2026 11:17:08 +0000 (0:00:00.257) 0:09:51.878 ***** 2026-01-21 11:17:08.477797 | controller | Wednesday 21 January 2026 11:17:08 +0000 (0:00:00.257) 0:09:51.877 ***** 2026-01-21 11:17:08.477812 | controller | ok: [controller] 2026-01-21 11:18:09.600753 | controller | 2026-01-21 11:18:09.600910 | controller | TASK [run_hook : Run hook without retry - Deploy watcher operator] ************* 2026-01-21 11:18:09.600929 | controller | Wednesday 21 January 2026 11:17:08 +0000 (0:00:00.270) 0:09:52.149 ***** 2026-01-21 11:18:09.600937 | controller | Wednesday 21 January 2026 11:17:08 +0000 (0:00:00.270) 0:09:52.147 ***** 2026-01-21 11:18:09.600963 | controller | Follow script's output here: /home/zuul/ci-framework-data/logs/ci_script_004_run_hook_without_retry_deploy.log 2026-01-21 11:18:09.633411 | controller | changed: [controller] 2026-01-21 11:18:09.633462 | controller | 2026-01-21 11:18:09.633471 | controller | TASK [run_hook : Run hook with retry - Deploy watcher operator] **************** 2026-01-21 11:18:09.633478 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:01:01.121) 0:10:53.271 ***** 2026-01-21 11:18:09.633485 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:01:01.121) 0:10:53.269 ***** 2026-01-21 11:18:09.633496 | controller | skipping: [controller] 2026-01-21 11:18:09.876879 | controller | 2026-01-21 11:18:09.876929 | controller | TASK [run_hook : Check if we have a file path={{ cifmw_basedir }}/artifacts/{{ step }}_{{ hook_name }}.yml] *** 2026-01-21 11:18:09.876939 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:00:00.033) 0:10:53.305 ***** 2026-01-21 11:18:09.876946 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:00:00.033) 0:10:53.303 ***** 2026-01-21 11:18:09.876958 | controller | ok: [controller] 2026-01-21 11:18:09.909919 | controller | 2026-01-21 11:18:09.909959 | controller | TASK [run_hook : Load generated content in main playbook file={{ cifmw_basedir }}/artifacts/{{ step }}_{{ hook_name }}.yml] *** 2026-01-21 11:18:09.909967 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:00:00.243) 0:10:53.548 ***** 2026-01-21 11:18:09.909974 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:00:00.243) 0:10:53.547 ***** 2026-01-21 11:18:09.909993 | controller | skipping: [controller] 2026-01-21 11:21:21.760649 | controller | 2026-01-21 11:21:21.761175 | controller | TASK [install kuttl test_suite dependencies chdir={{ ansible_user_dir }}/{{ operator_basedir }}, _raw_params=make kuttl-test-prep] *** 2026-01-21 11:21:21.761216 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:00:00.033) 0:10:53.581 ***** 2026-01-21 11:21:21.761265 | controller | Wednesday 21 January 2026 11:18:09 +0000 (0:00:00.033) 0:10:53.580 ***** 2026-01-21 11:21:21.761318 | controller | changed: [controller] 2026-01-21 11:21:21.961617 | controller | 2026-01-21 11:21:21.961703 | controller | PLAY RECAP ********************************************************************* 2026-01-21 11:21:21.961773 | controller | controller : ok=114 changed=41 unreachable=0 failed=0 skipped=79 rescued=0 ignored=1 2026-01-21 11:21:21.961786 | controller | 2026-01-21 11:21:21.961795 | controller | Wednesday 21 January 2026 11:21:21 +0000 (0:03:11.847) 0:14:05.429 ***** 2026-01-21 11:21:21.961810 | controller | =============================================================================== 2026-01-21 11:21:21.961823 | controller | install_yamls_makes : Run openstack_init ------------------------------ 346.72s 2026-01-21 11:21:21.961832 | controller | install kuttl test_suite dependencies --------------------------------- 191.85s 2026-01-21 11:21:21.961840 | controller | openshift_obs : Wait for observability operator deployment ------------- 71.49s 2026-01-21 11:21:21.961848 | controller | run_hook : Run hook without retry - Deploy watcher operator ------------ 61.12s 2026-01-21 11:21:21.961854 | controller | install_yamls_makes : Run download_tools ------------------------------- 39.49s 2026-01-21 11:21:21.961860 | controller | ci_setup : Install needed packages ------------------------------------- 37.76s 2026-01-21 11:21:21.961868 | controller | run_hook : Run hook without retry - Download needed tools -------------- 34.13s 2026-01-21 11:21:21.961876 | controller | repo_setup : Initialize python venv and install requirements ------------ 9.58s 2026-01-21 11:21:21.961884 | controller | ci_setup : Install openshift client ------------------------------------- 6.15s 2026-01-21 11:21:21.961892 | controller | openshift_setup : Create required namespaces ---------------------------- 2.41s 2026-01-21 11:21:21.961900 | controller | repo_setup : Get repo-setup repository ---------------------------------- 1.86s 2026-01-21 11:21:21.961909 | controller | install_ca : Update ca bundle ------------------------------------------- 1.85s 2026-01-21 11:21:21.961986 | controller | networking_mapper : Check for Networking Environment Definition file existence --- 1.61s 2026-01-21 11:21:21.962041 | controller | openshift_obs : Install cluster observability operator. ----------------- 1.41s 2026-01-21 11:21:21.962054 | controller | Gathering Facts --------------------------------------------------------- 1.39s 2026-01-21 11:21:21.962063 | controller | ci_setup : Manage directories ------------------------------------------- 1.38s 2026-01-21 11:21:21.962071 | controller | repo_setup : Make sure git-core package is installed -------------------- 1.28s 2026-01-21 11:21:21.962079 | controller | openshift_setup : Patch network operator -------------------------------- 1.26s 2026-01-21 11:21:21.962087 | controller | openshift_obs : Wait for observability-operator pod --------------------- 1.20s 2026-01-21 11:21:21.962095 | controller | openshift_setup : Gather network.operator info -------------------------- 1.15s 2026-01-21 11:21:21.962103 | controller | Wednesday 21 January 2026 11:21:21 +0000 (0:03:11.849) 0:14:05.429 ***** 2026-01-21 11:21:21.962110 | controller | =============================================================================== 2026-01-21 11:21:21.962118 | controller | install_yamls_makes --------------------------------------------------- 386.35s 2026-01-21 11:21:21.962126 | controller | ansible.builtin.command ----------------------------------------------- 192.34s 2026-01-21 11:21:21.962134 | controller | run_hook --------------------------------------------------------------- 99.65s 2026-01-21 11:21:21.962142 | controller | openshift_obs ---------------------------------------------------------- 74.10s 2026-01-21 11:21:21.962150 | controller | ci_setup --------------------------------------------------------------- 46.91s 2026-01-21 11:21:21.962158 | controller | repo_setup ------------------------------------------------------------- 21.34s 2026-01-21 11:21:21.962203 | controller | openshift_setup --------------------------------------------------------- 6.62s 2026-01-21 11:21:21.962212 | controller | openshift_login --------------------------------------------------------- 5.51s 2026-01-21 11:21:21.962219 | controller | install_yamls ----------------------------------------------------------- 4.25s 2026-01-21 11:21:21.962225 | controller | install_ca -------------------------------------------------------------- 2.43s 2026-01-21 11:21:21.962231 | controller | cifmw_setup ------------------------------------------------------------- 1.80s 2026-01-21 11:21:21.962238 | controller | networking_mapper ------------------------------------------------------- 1.71s 2026-01-21 11:21:21.962244 | controller | gather_facts ------------------------------------------------------------ 1.39s 2026-01-21 11:21:21.962251 | controller | discover_latest_image --------------------------------------------------- 0.47s 2026-01-21 11:21:21.962257 | controller | ansible.builtin.set_fact ------------------------------------------------ 0.20s 2026-01-21 11:21:21.962263 | controller | ansible.builtin.include_role -------------------------------------------- 0.12s 2026-01-21 11:21:21.962270 | controller | ansible.builtin.include_vars -------------------------------------------- 0.08s 2026-01-21 11:21:21.962276 | controller | ansible.builtin.assert -------------------------------------------------- 0.08s 2026-01-21 11:21:21.962283 | controller | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2026-01-21 11:21:21.962289 | controller | total ----------------------------------------------------------------- 845.37s 2026-01-21 11:21:22.321320 | controller | changed 2026-01-21 11:21:22.399099 | 2026-01-21 11:21:22.399226 | PLAY RECAP 2026-01-21 11:21:22.399285 | controller | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-01-21 11:21:22.399325 | 2026-01-21 11:21:22.505076 | PRE-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/kuttl/kuttl-from-operator-deps.yaml@main] 2026-01-21 11:21:22.515726 | RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/kuttl/kuttl-from-operator-run.yaml@main] 2026-01-21 11:21:23.183742 | 2026-01-21 11:21:23.183856 | PLAY [Run ci/playbooks/kuttl/run-kuttl-from-operator-targets.yaml] 2026-01-21 11:21:23.208133 | 2026-01-21 11:21:23.208240 | TASK [Run kuttl tests from operator playbook] 2026-01-21 11:21:24.619046 | controller | 2026-01-21 11:21:24.690127 | controller | PLAY [controller] ************************************************************** 2026-01-21 11:21:24.690231 | controller | 2026-01-21 11:21:24.690239 | controller | TASK [Run hooks before running kuttl tests name=run_hook] ********************** 2026-01-21 11:21:24.690246 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.057) 0:00:00.057 ***** 2026-01-21 11:21:24.690252 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.055) 0:00:00.055 ***** 2026-01-21 11:21:24.690271 | controller | 2026-01-21 11:21:24.808654 | controller | TASK [run_hook : Assert parameters are valid quiet=True, that=['_list_hooks is not string', '_list_hooks is not mapping', '_list_hooks is iterable', '(hooks | default([])) is not string', '(hooks | default([])) is not mapping', '(hooks | default([])) is iterable']] *** 2026-01-21 11:21:24.808782 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.070) 0:00:00.128 ***** 2026-01-21 11:21:24.808793 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.070) 0:00:00.126 ***** 2026-01-21 11:21:24.808815 | controller | ok: [controller] 2026-01-21 11:21:24.952907 | controller | 2026-01-21 11:21:24.953018 | controller | TASK [run_hook : Assert single hooks are all mappings quiet=True, that=['_not_mapping_hooks | length == 0'], msg=All single hooks must be a list of mappings or a mapping.] *** 2026-01-21 11:21:24.953067 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.118) 0:00:00.247 ***** 2026-01-21 11:21:24.953079 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.118) 0:00:00.245 ***** 2026-01-21 11:21:24.953126 | controller | ok: [controller] 2026-01-21 11:21:25.217621 | controller | 2026-01-21 11:21:25.217782 | controller | TASK [run_hook : Loop on hooks for pre_kuttl_from_operator _raw_params={{ hook.type }}.yml] *** 2026-01-21 11:21:25.217799 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.144) 0:00:00.391 ***** 2026-01-21 11:21:25.217808 | controller | Wednesday 21 January 2026 11:21:24 +0000 (0:00:00.144) 0:00:00.389 ***** 2026-01-21 11:21:25.217832 | controller | skipping: [controller] 2026-01-21 11:38:24.296808 | controller | 2026-01-21 11:38:24.297175 | controller | TASK [run kuttl test suite from operator Makefile chdir={{ ansible_user_dir }}/{{ operator_basedir }}, _raw_params=make kuttl-test-run] *** 2026-01-21 11:38:24.297202 | controller | Wednesday 21 January 2026 11:21:25 +0000 (0:00:00.264) 0:00:00.656 ***** 2026-01-21 11:38:24.297214 | controller | Wednesday 21 January 2026 11:21:25 +0000 (0:00:00.264) 0:00:00.654 ***** 2026-01-21 11:38:24.297242 | controller | task path: /home/zuul/src/github.com/openstack-k8s-operators/ci-framework/ci/playbooks/kuttl/run-kuttl-from-operator-targets.yaml:16 2026-01-21 11:38:24.297300 | controller | fatal: [controller]: FAILED! => 2026-01-21 11:38:24.297323 | controller | changed: true 2026-01-21 11:38:24.297329 | controller | cmd: 2026-01-21 11:38:24.297336 | controller | - make 2026-01-21 11:38:24.297342 | controller | - kuttl-test-run 2026-01-21 11:38:24.297348 | controller | delta: '0:16:58.420747' 2026-01-21 11:38:24.297373 | controller | end: '2026-01-21 11:38:24.014293' 2026-01-21 11:38:24.297383 | controller | msg: non-zero return code 2026-01-21 11:38:24.297391 | controller | rc: 2 2026-01-21 11:38:24.297398 | controller | start: '2026-01-21 11:21:25.593546' 2026-01-21 11:38:24.297406 | controller | stderr: |- 2026-01-21 11:38:24.297414 | controller | Warning: spec.privateKey.rotationPolicy: In cert-manager >= v1.18.0, the default value changed from `Never` to `Always`. 2026-01-21 11:38:24.297422 | controller | Warning: The 'cinderBackup' field is deprecated and will be removed in a future release. Please migrate to 'cinderBackups'. 2026-01-21 11:38:24.297429 | controller | make: *** [Makefile:444: kuttl-test-run] Error 1 2026-01-21 11:38:24.297445 | controller | stderr_lines: 2026-01-21 11:38:24.297454 | controller | - 'Warning: spec.privateKey.rotationPolicy: In cert-manager >= v1.18.0, the default 2026-01-21 11:38:24.297462 | controller | value changed from `Never` to `Always`.' 2026-01-21 11:38:24.297470 | controller | - 'Warning: The ''cinderBackup'' field is deprecated and will be removed in a future 2026-01-21 11:38:24.297477 | controller | release. Please migrate to ''cinderBackups''.' 2026-01-21 11:38:24.297485 | controller | - 'make: *** [Makefile:444: kuttl-test-run] Error 1' 2026-01-21 11:38:24.297509 | controller | stdout: "oc kuttl test --v 1 --start-kind=false --config test/kuttl/test-suites/default/config.yaml\n=== 2026-01-21 11:38:24.297518 | controller | RUN kuttl\n harness.go:463: starting setup\n harness.go:255: running tests 2026-01-21 11:38:24.297526 | controller | using configured kubeconfig.\n harness.go:278: Successful connection to cluster 2026-01-21 11:38:24.297534 | controller | at: https://api.crc.testing:6443\n harness.go:363: running tests\n harness.go:75: 2026-01-21 11:38:24.297544 | controller | going to run test suite with timeout of 300 seconds for each step\n harness.go:375: 2026-01-21 11:38:24.297552 | controller | testsuite: test/kuttl/test-suites/default/ has 10 tests\n=== RUN kuttl/harness\n=== 2026-01-21 11:38:24.297561 | controller | RUN kuttl/harness/common\n=== PAUSE kuttl/harness/common\n=== RUN kuttl/harness/deps\n=== 2026-01-21 11:38:24.297568 | controller | PAUSE kuttl/harness/deps\n=== RUN kuttl/harness/watcher\n=== PAUSE kuttl/harness/watcher\n=== 2026-01-21 11:38:24.297576 | controller | RUN kuttl/harness/watcher-api-scaling\n=== PAUSE kuttl/harness/watcher-api-scaling\n=== 2026-01-21 11:38:24.297584 | controller | RUN kuttl/harness/watcher-cinder\n=== PAUSE kuttl/harness/watcher-cinder\n=== 2026-01-21 11:38:24.297591 | controller | RUN kuttl/harness/watcher-notification\n=== PAUSE kuttl/harness/watcher-notification\n=== 2026-01-21 11:38:24.297597 | controller | RUN kuttl/harness/watcher-rmquser\n=== PAUSE kuttl/harness/watcher-rmquser\n=== 2026-01-21 11:38:24.297604 | controller | RUN kuttl/harness/watcher-tls\n=== PAUSE kuttl/harness/watcher-tls\n=== RUN kuttl/harness/watcher-tls-certs-change\n=== 2026-01-21 11:38:24.297611 | controller | PAUSE kuttl/harness/watcher-tls-certs-change\n=== RUN kuttl/harness/watcher-topology\n=== 2026-01-21 11:38:24.297619 | controller | PAUSE kuttl/harness/watcher-topology\n=== CONT kuttl/harness/common\n logger.go:42: 2026-01-21 11:38:24.297627 | controller | 11:21:26 | common | Ignoring cleanup-assert.yaml as it does not match file name 2026-01-21 11:38:24.297635 | controller | regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n logger.go:42: 11:21:26 | common | 2026-01-21 11:38:24.297642 | controller | Ignoring cleanup-errors.yaml as it does not match file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n 2026-01-21 11:38:24.297657 | controller | \ logger.go:42: 11:21:26 | common | Ignoring cleanup-watcher.yaml as it does not 2026-01-21 11:38:24.297665 | controller | match file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n logger.go:42: 11:21:26 2026-01-21 11:38:24.297672 | controller | | common | Ignoring deploy-with-defaults.yaml as it does not match file name regexp: 2026-01-21 11:38:24.297679 | controller | ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n logger.go:42: 11:21:26 | common | Skipping 2026-01-21 11:38:24.297686 | controller | creation of user-supplied namespace: watcher-kuttl-default\n logger.go:42: 11:21:26 2026-01-21 11:38:24.297694 | controller | | common | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-topology\n 2026-01-21 11:38:24.297701 | controller | \ logger.go:42: 11:21:26 | watcher-topology | Skipping creation of user-supplied 2026-01-21 11:38:24.297708 | controller | namespace: watcher-kuttl-default\n logger.go:42: 11:21:26 | watcher-topology/0-cleanup-watcher 2026-01-21 11:38:24.297744 | controller | | starting test step 0-cleanup-watcher\n logger.go:42: 11:21:26 | watcher-topology/0-cleanup-watcher 2026-01-21 11:38:24.297752 | controller | | test step completed 0-cleanup-watcher\n logger.go:42: 11:21:26 | watcher-topology/1-deploy-with-topology 2026-01-21 11:38:24.297760 | controller | | starting test step 1-deploy-with-topology\n logger.go:42: 11:21:26 | watcher-topology/1-deploy-with-topology 2026-01-21 11:38:24.297767 | controller | | Topology:watcher-kuttl-default/watcher-api created\n logger.go:42: 11:21:26 2026-01-21 11:38:24.297775 | controller | | watcher-topology/1-deploy-with-topology | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.297782 | controller | created\n logger.go:42: 11:22:33 | watcher-topology/1-deploy-with-topology | 2026-01-21 11:38:24.297797 | controller | test step completed 1-deploy-with-topology\n logger.go:42: 11:22:33 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297812 | controller | | starting test step 2-cleanup-watcher\n logger.go:42: 11:22:34 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297819 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.297827 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.297835 | controller | ]\n ]\n logger.go:42: 11:22:34 | watcher-topology/2-cleanup-watcher | 2026-01-21 11:38:24.297842 | controller | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.297850 | controller | value -c Name -c Type\n logger.go:42: 11:22:34 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297857 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:22:38 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297865 | controller | | + '[' 1 == 0 ']'\n logger.go:42: 11:22:39 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297872 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.297879 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.297889 | controller | ]\n ]\n logger.go:42: 11:22:39 | watcher-topology/2-cleanup-watcher | 2026-01-21 11:38:24.297895 | controller | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.297901 | controller | value -c Name -c Type\n logger.go:42: 11:22:39 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297907 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:22:42 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297913 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:22:43 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297919 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.297925 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.297931 | controller | ]\n ]\n logger.go:42: 11:22:43 | watcher-topology/2-cleanup-watcher | 2026-01-21 11:38:24.297937 | controller | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.297943 | controller | value -c Name -c Type\n logger.go:42: 11:22:43 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297949 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:22:46 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297955 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:22:46 | watcher-topology/2-cleanup-watcher 2026-01-21 11:38:24.297961 | controller | | test step completed 2-cleanup-watcher\n logger.go:42: 11:22:46 | watcher-topology 2026-01-21 11:38:24.297967 | controller | | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-tls-certs-change\n 2026-01-21 11:38:24.297973 | controller | \ logger.go:42: 11:22:47 | watcher-tls-certs-change | Skipping creation of user-supplied 2026-01-21 11:38:24.297978 | controller | namespace: watcher-kuttl-default\n logger.go:42: 11:22:47 | watcher-tls-certs-change/0-cleanup-watcher 2026-01-21 11:38:24.297984 | controller | | starting test step 0-cleanup-watcher\n logger.go:42: 11:22:47 | watcher-tls-certs-change/0-cleanup-watcher 2026-01-21 11:38:24.297990 | controller | | test step completed 0-cleanup-watcher\n logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.297996 | controller | | starting test step 1-deploy-with-tlse\n logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298002 | controller | | Certificate:watcher-kuttl-default/watcher-internal-svc created\n logger.go:42: 2026-01-21 11:38:24.298008 | controller | 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | Certificate:watcher-kuttl-default/watcher-public-svc 2026-01-21 11:38:24.298014 | controller | created\n logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298021 | controller | | Watcher:watcher-kuttl-default/watcher-kuttl created\n logger.go:42: 11:22:47 2026-01-21 11:38:24.298031 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c set -euxo 2026-01-21 11:38:24.298037 | controller | pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.298043 | controller | cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc get -n $NAMESPACE 2026-01-21 11:38:24.298049 | controller | secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n 2026-01-21 11:38:24.298055 | controller | \ # ensure that the svc secret and cert secret match\n if [ \"${public_svc_cert}\" 2026-01-21 11:38:24.298061 | controller | != \"${public_secret_cert}\" ]; then\n exit 1\n fi\n \n internal_svc_cert=$(oc 2026-01-21 11:38:24.298067 | controller | rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n 2026-01-21 11:38:24.298073 | controller | \ internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.298079 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.298088 | controller | secret and cert secret match\n if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" 2026-01-21 11:38:24.298094 | controller | ]; then\n exit 1\n fi\n ]\n logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298100 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298106 | controller | \ logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298112 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298119 | controller | 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298125 | controller | 11:22:48 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298131 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298138 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298143 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298153 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298159 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298165 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298171 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298177 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298182 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298191 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298197 | controller | 1\n fi\n ]\n logger.go:42: 11:22:48 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298203 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298209 | controller | \ logger.go:42: 11:22:49 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298215 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298221 | controller | 11:22:49 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298228 | controller | 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298235 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298245 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298251 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298259 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298266 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298272 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298280 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298286 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298292 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298298 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298304 | controller | 1\n fi\n ]\n logger.go:42: 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298310 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298316 | controller | \ logger.go:42: 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298322 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298328 | controller | 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298334 | controller | 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298340 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298346 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298352 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298358 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298367 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298373 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298379 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298385 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298391 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298397 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298403 | controller | 1\n fi\n ]\n logger.go:42: 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298409 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298415 | controller | \ logger.go:42: 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298421 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298427 | controller | 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298432 | controller | 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298438 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298447 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298457 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298464 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298469 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298475 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298481 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298487 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298493 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298499 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298505 | controller | 1\n fi\n ]\n logger.go:42: 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298511 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298517 | controller | \ logger.go:42: 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298523 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298529 | controller | 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298535 | controller | 11:22:53 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298541 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298576 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298582 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298588 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298594 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298600 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298606 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298612 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298618 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298624 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298630 | controller | 1\n fi\n ]\n logger.go:42: 11:22:53 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298636 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298642 | controller | \ logger.go:42: 11:22:54 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298648 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298654 | controller | 11:22:54 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298659 | controller | 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298669 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298675 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298681 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298696 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298702 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298708 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298733 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298740 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298746 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298752 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298758 | controller | 1\n fi\n ]\n logger.go:42: 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298764 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298770 | controller | \ logger.go:42: 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298776 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298782 | controller | 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298792 | controller | 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298798 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298804 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298810 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298815 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298821 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298827 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298833 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298839 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298845 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298851 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298857 | controller | 1\n fi\n ]\n logger.go:42: 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298863 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298869 | controller | \ logger.go:42: 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298875 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298881 | controller | 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298887 | controller | 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298893 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298899 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.298905 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298911 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298917 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298927 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.298933 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.298939 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.298945 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.298954 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.298960 | controller | 1\n fi\n ]\n logger.go:42: 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.298969 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.298976 | controller | \ logger.go:42: 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.298982 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.298987 | controller | 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.298993 | controller | 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.298999 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299005 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299011 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299017 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299023 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299029 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299035 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299041 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299047 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299053 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299059 | controller | 1\n fi\n ]\n logger.go:42: 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299065 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299073 | controller | \ logger.go:42: 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.299080 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.299085 | controller | 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299091 | controller | 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299097 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299103 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299109 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299115 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299121 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299127 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299137 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299143 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299149 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299155 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299161 | controller | 1\n fi\n ]\n logger.go:42: 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299167 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299173 | controller | \ logger.go:42: 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.299180 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.299187 | controller | 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299193 | controller | 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299199 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299205 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299211 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299217 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299223 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299229 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299235 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299243 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299250 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299255 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299261 | controller | 1\n fi\n ]\n logger.go:42: 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299267 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299273 | controller | \ logger.go:42: 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.299281 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.299287 | controller | 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299293 | controller | 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299299 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299305 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299311 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299316 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299322 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299328 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299334 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299347 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299353 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299359 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299365 | controller | 1\n fi\n ]\n logger.go:42: 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299371 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299377 | controller | \ logger.go:42: 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.299383 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.299389 | controller | 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299394 | controller | 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299400 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299406 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299412 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299418 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299424 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299430 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299436 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299442 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299447 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299453 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299459 | controller | 1\n fi\n ]\n logger.go:42: 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299465 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299471 | controller | \ logger.go:42: 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.299477 | controller | from server (NotFound): pods \"watcher-kuttl-api-0\" not found\n logger.go:42: 2026-01-21 11:38:24.299483 | controller | 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299489 | controller | 11:23:05 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299495 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299501 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299507 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299516 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299522 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299528 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299533 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299539 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299545 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299555 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299561 | controller | 1\n fi\n ]\n logger.go:42: 11:23:05 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299567 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299573 | controller | \ logger.go:42: 11:23:06 | watcher-tls-certs-change/1-deploy-with-tlse | error: 2026-01-21 11:38:24.299579 | controller | unable to upgrade connection: container not found (\"watcher-api\")\n logger.go:42: 2026-01-21 11:38:24.299585 | controller | 11:23:06 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299591 | controller | 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299597 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299603 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299609 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299614 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299623 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299629 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299635 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299804 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299811 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299822 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299828 | controller | 1\n fi\n ]\n logger.go:42: 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299834 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299840 | controller | \ logger.go:42: 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | error: 2026-01-21 11:38:24.299845 | controller | unable to upgrade connection: container not found (\"watcher-api\")\n logger.go:42: 2026-01-21 11:38:24.299851 | controller | 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=\n logger.go:42: 2026-01-21 11:38:24.299857 | controller | 11:23:08 | watcher-tls-certs-change/1-deploy-with-tlse | running command: [sh -c 2026-01-21 11:38:24.299863 | controller | set -euxo pipefail\n public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299869 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n public_secret_cert=$(oc 2026-01-21 11:38:24.299875 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299881 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299887 | controller | \ if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299893 | controller | 1\n fi\n \n internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api 2026-01-21 11:38:24.299898 | controller | watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n internal_secret_cert=$(oc 2026-01-21 11:38:24.299904 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.299910 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.299916 | controller | \ if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" ]; then\n exit 2026-01-21 11:38:24.299927 | controller | 1\n fi\n ]\n logger.go:42: 11:23:08 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299933 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.299939 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert='-----BEGIN 2026-01-21 11:38:24.299945 | controller | CERTIFICATE-----\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.299963 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.299969 | controller | 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.299975 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.299994 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.300000 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.300006 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.300012 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.300018 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.300024 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.300030 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.300044 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.300050 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.300056 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.300062 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.300068 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.300073 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.300079 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.300085 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.300091 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.300097 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.300104 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.300114 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300120 | controller | | ++ base64 --decode\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300126 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.300132 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert='-----BEGIN 2026-01-21 11:38:24.300138 | controller | CERTIFICATE-----\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300144 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.300150 | controller | 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.300156 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.300162 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.300168 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.300182 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.300189 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.300195 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.300205 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.300211 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.300217 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.300223 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.300230 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.300237 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.300243 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.300249 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.300256 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.300263 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.300270 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.300277 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.300289 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.300295 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300301 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300307 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.300313 | controller | 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.300318 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.300324 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.300330 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.300336 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.300342 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.300348 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.300354 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.300360 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.300366 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.300379 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.300392 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.300398 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.300404 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.300410 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.300416 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.300422 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.300428 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.300434 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.300440 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.300446 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:09 2026-01-21 11:38:24.300456 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.300462 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.300468 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.300474 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.300480 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.300486 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.300492 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.300498 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.300503 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.300509 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.300515 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.300521 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.300527 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.300533 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.300539 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.300548 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.300554 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.300568 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.300574 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.300580 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.300586 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.300592 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300598 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.300604 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert='-----BEGIN 2026-01-21 11:38:24.300614 | controller | CERTIFICATE-----\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300621 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.300627 | controller | 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.300639 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.300647 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.300654 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.300662 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.300670 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.300677 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.300684 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.300691 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.300698 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.300704 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.300754 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.300764 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.300770 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.300776 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.300782 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.300788 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.300794 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.300806 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.300813 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.300819 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300825 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.300831 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.300866 | controller | --decode\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.300874 | controller | | + internal_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:09 2026-01-21 11:38:24.300880 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.300886 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.300892 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.300898 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.300904 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.300910 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.300916 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.300922 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.300928 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.300934 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.300940 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.300946 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.300952 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.300958 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.300964 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.300970 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.300980 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.300986 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.300992 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.300998 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.301004 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.301010 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301016 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301025 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.301031 | controller | 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.301037 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.301043 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.301049 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.301066 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.301072 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.301078 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.301085 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.301090 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.301096 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.301102 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.301108 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.301114 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.301120 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.301126 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.301132 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.301138 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.301144 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.301150 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.301156 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.301165 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:09 2026-01-21 11:38:24.301172 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.301179 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.301185 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.301195 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.301201 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.301206 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.301212 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.301218 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.301227 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.301233 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.301239 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.301245 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.301267 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.301274 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.301280 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.301286 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.301292 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.301298 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.301303 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.301310 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.301316 | controller | \ logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.301322 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301328 | controller | | running command: [sh -c set -euxo pipefail\n public_svc_cert=$(oc rsh -n 2026-01-21 11:38:24.301334 | controller | $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n 2026-01-21 11:38:24.301340 | controller | \ public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.301346 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.301352 | controller | secret and cert secret match\n if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" 2026-01-21 11:38:24.301358 | controller | ]; then\n exit 1\n fi\n \n internal_svc_cert=$(oc 2026-01-21 11:38:24.301367 | controller | rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n 2026-01-21 11:38:24.301373 | controller | \ internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.301378 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.301384 | controller | secret and cert secret match\n if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" 2026-01-21 11:38:24.301390 | controller | ]; then\n exit 1\n fi\n ]\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301396 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.301402 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert='-----BEGIN 2026-01-21 11:38:24.301408 | controller | CERTIFICATE-----\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301421 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.301427 | controller | 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.301433 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.301439 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.301444 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.301450 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.301456 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.301463 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.301468 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.301481 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.301487 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.301493 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.301503 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.301509 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.301514 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.301520 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.301526 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.301532 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.301542 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.301548 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.301554 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.301560 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301566 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.301572 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.301578 | controller | --decode\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301584 | controller | | + public_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:11 2026-01-21 11:38:24.301609 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.301615 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.301621 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.301627 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.301633 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.301639 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.301645 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.301651 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.301657 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.301663 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.301669 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.301675 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.301681 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.301693 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.301700 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.301729 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.301738 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.301751 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.301757 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.301763 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.301769 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.301775 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301781 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.301787 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.301793 | controller | 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.301799 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.301805 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.301811 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.301817 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.301823 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.301829 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.301835 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.301841 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.301847 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.301852 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.301858 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.301864 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.301870 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.301876 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.301882 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.301888 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.301894 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.301913 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.301927 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.301933 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:11 2026-01-21 11:38:24.301939 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.301945 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.301951 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.301957 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.301963 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.301969 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.301975 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.301981 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.301987 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.301993 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.301999 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.302004 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.302010 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.302016 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.302022 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.302028 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.302034 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.302040 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.302046 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.302054 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.302060 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.302070 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302078 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.302086 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert='-----BEGIN 2026-01-21 11:38:24.302105 | controller | CERTIFICATE-----\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302114 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.302121 | controller | 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.302129 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.302147 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.302155 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.302170 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.302176 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.302183 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.302189 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.302194 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.302200 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.302206 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.302212 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.302218 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.302224 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.302232 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.302238 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.302245 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.302251 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.302257 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.302265 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.302277 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302285 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.302291 | controller | \ logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.302297 | controller | --decode\n logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302303 | controller | | + internal_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:12 2026-01-21 11:38:24.302309 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.302315 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.302321 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.302327 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.302333 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.302339 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.302345 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.302359 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.302374 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.302380 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.302387 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.302393 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.302399 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.302404 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.302410 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.302416 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.302422 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.302428 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.302434 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.302440 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.302450 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.302456 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302462 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302468 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.302474 | controller | 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.302480 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.302486 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.302492 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.302498 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.302504 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.302510 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.302516 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.302522 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.302528 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.302534 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.302542 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.302565 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.302580 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.302586 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.302592 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.302598 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.302604 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.302610 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.302616 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.302625 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:12 2026-01-21 11:38:24.302632 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.302638 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.302643 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.302649 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.302655 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.302661 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.302667 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.302673 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.302679 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.302685 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.302691 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.302697 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.302703 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.302729 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.302739 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.302755 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.302761 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.302767 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.302773 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.302780 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.302786 | controller | \ logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.302801 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302808 | controller | | running command: [sh -c set -euxo pipefail\n public_svc_cert=$(oc rsh -n 2026-01-21 11:38:24.302814 | controller | $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n 2026-01-21 11:38:24.302830 | controller | \ public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.302837 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.302843 | controller | secret and cert secret match\n if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" 2026-01-21 11:38:24.302849 | controller | ]; then\n exit 1\n fi\n \n internal_svc_cert=$(oc 2026-01-21 11:38:24.302855 | controller | rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n 2026-01-21 11:38:24.302861 | controller | \ internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.302867 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.302873 | controller | secret and cert secret match\n if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" 2026-01-21 11:38:24.302879 | controller | ]; then\n exit 1\n fi\n ]\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302885 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.302891 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert='-----BEGIN 2026-01-21 11:38:24.302897 | controller | CERTIFICATE-----\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.302903 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.302909 | controller | 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.302915 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.302921 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.302927 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.302933 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.302939 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.302945 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.302951 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.302957 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.302963 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.302969 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.302975 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.302981 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.302987 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.302996 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.303002 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.303008 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.303023 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.303036 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.303042 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.303049 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303055 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.303061 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.303067 | controller | --decode\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303073 | controller | | + public_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:13 2026-01-21 11:38:24.303079 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.303085 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.303091 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.303097 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.303103 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.303109 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.303115 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.303121 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.303127 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.303133 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.303139 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.303145 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.303151 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.303157 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.303166 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.303172 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.303179 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.303187 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.303195 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.303203 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.303211 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.303219 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303227 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303234 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.303252 | controller | 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.303260 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.303268 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.303275 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.303286 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.303293 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.303301 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.303309 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.303317 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.303325 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.303333 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.303341 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.303348 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.303356 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.303365 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.303378 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.303386 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.303394 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.303402 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.303409 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.303416 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:13 2026-01-21 11:38:24.303422 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.303429 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.303435 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.303441 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.303447 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.303453 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.303459 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.303482 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.303514 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.303523 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.303532 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.303539 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.303547 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.303555 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.303562 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.303568 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.303574 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.303580 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.303590 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.303596 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.303603 | controller | \ logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.303609 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303615 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.303621 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert='-----BEGIN 2026-01-21 11:38:24.303627 | controller | CERTIFICATE-----\n logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303633 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.303639 | controller | 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.303645 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.303651 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.303657 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.303663 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.303669 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.303675 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.303683 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.303699 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.303728 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.303743 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.303771 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.303789 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.303798 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.303813 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.303826 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.303835 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.303850 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.303858 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.303866 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.303874 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303881 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.303889 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.303896 | controller | --decode\n logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.303903 | controller | | + internal_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:14 2026-01-21 11:38:24.303911 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.303919 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.303927 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.303935 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.303943 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.303951 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.303959 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.303967 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.303974 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.303981 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.303989 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.303996 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.304003 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.304011 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.304019 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.304044 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.304065 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.304078 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.304087 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.304095 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.304102 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.304110 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304117 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304124 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.304132 | controller | 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.304139 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.304145 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.304151 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.304159 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.304167 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.304174 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.304183 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.304192 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.304200 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.304207 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.304215 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.304223 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.304230 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.304238 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.304245 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.304253 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.304266 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.304277 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.304285 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.304292 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:14 2026-01-21 11:38:24.304300 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.304319 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.304328 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.304336 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.304344 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.304352 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.304359 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.304367 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.304374 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.304382 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.304390 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.304397 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.304405 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.304412 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.304420 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.304428 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.304435 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.304449 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.304456 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.304464 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.304479 | controller | \ logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.304486 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304493 | controller | | running command: [sh -c set -euxo pipefail\n public_svc_cert=$(oc rsh -n 2026-01-21 11:38:24.304499 | controller | $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n 2026-01-21 11:38:24.304505 | controller | \ public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.304511 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.304517 | controller | secret and cert secret match\n if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" 2026-01-21 11:38:24.304523 | controller | ]; then\n exit 1\n fi\n \n internal_svc_cert=$(oc 2026-01-21 11:38:24.304529 | controller | rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n 2026-01-21 11:38:24.304535 | controller | \ internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.304542 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.304548 | controller | secret and cert secret match\n if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" 2026-01-21 11:38:24.304554 | controller | ]; then\n exit 1\n fi\n ]\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304560 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.304576 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert='-----BEGIN 2026-01-21 11:38:24.304582 | controller | CERTIFICATE-----\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304588 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.304594 | controller | 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.304600 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.304607 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.304613 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.304629 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.304636 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.304642 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.304648 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.304653 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.304659 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.304665 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.304675 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.304681 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.304687 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.304693 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.304699 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.304705 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.304727 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.304737 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.304743 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.304750 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304755 | controller | | ++ base64 --decode\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304770 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.304776 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert='-----BEGIN 2026-01-21 11:38:24.304782 | controller | CERTIFICATE-----\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304788 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.304794 | controller | 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.304799 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.304815 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.304821 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.304827 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.304834 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.304841 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.304849 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.304855 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.304861 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.304867 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.304876 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.304882 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.304888 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.304894 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.304900 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.304906 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.304912 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.304918 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.304924 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.304931 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304937 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.304943 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.304949 | controller | 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.304955 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.304961 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.304967 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.304972 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.304978 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.304984 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.304990 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.305008 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.305014 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.305020 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.305028 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.305037 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.305043 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.305049 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.305055 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.305061 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.305067 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.305073 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.305078 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.305084 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:15 2026-01-21 11:38:24.305090 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.305096 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.305102 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.305108 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.305113 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.305119 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.305125 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.305131 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.305137 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.305142 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.305148 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.305154 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.305160 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.305166 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.305172 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.305182 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.305196 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.305202 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.305208 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.305214 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.305220 | controller | \ logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.305227 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305234 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.305249 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert='-----BEGIN 2026-01-21 11:38:24.305256 | controller | CERTIFICATE-----\n logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305263 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.305269 | controller | 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.305276 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.305284 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.305289 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.305295 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.305301 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.305307 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.305313 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.305319 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.305325 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.305331 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.305337 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.305343 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.305349 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.305358 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.305364 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.305370 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.305376 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.305389 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.305395 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.305401 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305416 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.305422 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.305428 | controller | --decode\n logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305434 | controller | | + internal_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:16 2026-01-21 11:38:24.305440 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.305446 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.305452 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.305458 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.305464 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.305470 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.305476 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.305482 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.305488 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.305494 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.305500 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.305506 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.305511 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.305517 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.305526 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.305532 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.305538 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.305544 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.305550 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.305556 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.305562 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.305567 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305573 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305579 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.305585 | controller | 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.305591 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.305597 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.305610 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.305624 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.305630 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.305636 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.305642 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.305648 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.305654 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.305660 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.305666 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.305672 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.305678 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.305683 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.305693 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.305699 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.305705 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.305724 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.305733 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.305739 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:16 2026-01-21 11:38:24.305745 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.305751 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.305757 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.305763 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.305768 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.305774 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.305780 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.305786 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.305800 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.305806 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.305820 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.305826 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.305832 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.305838 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.305845 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.305850 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.305856 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.305862 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.305872 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.305878 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.305884 | controller | \ logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.305890 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305896 | controller | | running command: [sh -c set -euxo pipefail\n public_svc_cert=$(oc rsh -n 2026-01-21 11:38:24.305901 | controller | $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n 2026-01-21 11:38:24.305907 | controller | \ public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.305913 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.305919 | controller | secret and cert secret match\n if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" 2026-01-21 11:38:24.305925 | controller | ]; then\n exit 1\n fi\n \n internal_svc_cert=$(oc 2026-01-21 11:38:24.305931 | controller | rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n 2026-01-21 11:38:24.305937 | controller | \ internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.305942 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.305948 | controller | secret and cert secret match\n if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" 2026-01-21 11:38:24.305954 | controller | ]; then\n exit 1\n fi\n ]\n logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305960 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.305966 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert='-----BEGIN 2026-01-21 11:38:24.305972 | controller | CERTIFICATE-----\n logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.305978 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.305984 | controller | 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.305990 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.305996 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.306002 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.306008 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.306014 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.306020 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.306034 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.306040 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.306049 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.306055 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.306069 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.306076 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.306082 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.306087 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.306093 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.306099 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.306105 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.306111 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.306117 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.306123 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306129 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.306135 | controller | \ logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.306141 | controller | --decode\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306147 | controller | | + public_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:18 2026-01-21 11:38:24.306153 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.306159 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.306165 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.306171 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.306178 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.306185 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.306191 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.306197 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.306203 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.306212 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.306217 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.306224 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.306239 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.306245 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.306251 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.306257 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.306263 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.306269 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.306275 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.306281 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.306288 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.306294 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306300 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306306 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.306313 | controller | 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.306318 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.306324 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.306330 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.306336 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.306342 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.306347 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.306353 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.306359 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.306368 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.306374 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.306380 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.306386 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.306392 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.306398 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.306404 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.306410 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.306422 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.306436 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.306442 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.306448 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:18 2026-01-21 11:38:24.306453 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.306459 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.306465 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.306471 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.306477 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.306483 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.306489 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.306495 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.306501 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.306507 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.306513 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.306519 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.306528 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.306534 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.306540 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.306586 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.306604 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.306612 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.306619 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.306626 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.306634 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.306641 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306649 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.306656 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert='-----BEGIN 2026-01-21 11:38:24.306663 | controller | CERTIFICATE-----\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306670 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.306678 | controller | 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.306697 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.306704 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.306725 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.306739 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.306745 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.306751 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.306757 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.306763 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.306769 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.306775 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.306787 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.306793 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.306800 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.306806 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.306812 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.306818 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.306824 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.306830 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.306836 | controller | \ logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.306842 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306848 | controller | | ++ base64 --decode\n logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306855 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.306861 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert='-----BEGIN 2026-01-21 11:38:24.306867 | controller | CERTIFICATE-----\n logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.306873 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.306878 | controller | 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.306884 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.306891 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.306897 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.306903 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.306918 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.306925 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.306931 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.306939 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.306946 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.306955 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.306961 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.306967 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.306973 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.306979 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.306985 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.306991 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.306997 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.307002 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.307008 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.307014 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307020 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307026 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.307032 | controller | 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.307038 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.307043 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.307049 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.307055 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.307061 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.307067 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.307073 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.307079 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.307085 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.307091 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.307107 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.307120 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.307126 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.307132 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.307137 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.307143 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.307149 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.307165 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.307171 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.307177 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:19 2026-01-21 11:38:24.307183 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.307189 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.307195 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.307201 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.307207 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.307212 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.307218 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.307224 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.307232 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.307238 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.307245 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.307250 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.307257 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.307264 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.307271 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.307282 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.307296 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.307302 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.307308 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.307323 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.307329 | controller | \ logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.307335 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307341 | controller | | running command: [sh -c set -euxo pipefail\n public_svc_cert=$(oc rsh -n 2026-01-21 11:38:24.307347 | controller | $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n 2026-01-21 11:38:24.307353 | controller | \ public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.307359 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.307365 | controller | secret and cert secret match\n if [ \"${public_svc_cert}\" != \"${public_secret_cert}\" 2026-01-21 11:38:24.307371 | controller | ]; then\n exit 1\n fi\n \n internal_svc_cert=$(oc 2026-01-21 11:38:24.307377 | controller | rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n 2026-01-21 11:38:24.307383 | controller | \ internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.307389 | controller | -o jsonpath='{.data.tls\\.crt}' | base64 --decode)\n # ensure that the svc 2026-01-21 11:38:24.307395 | controller | secret and cert secret match\n if [ \"${internal_svc_cert}\" != \"${internal_secret_cert}\" 2026-01-21 11:38:24.307401 | controller | ]; then\n exit 1\n fi\n ]\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307407 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.307413 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert='-----BEGIN 2026-01-21 11:38:24.307419 | controller | CERTIFICATE-----\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307425 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.307431 | controller | 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.307437 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.307443 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.307449 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.307455 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.307461 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.307470 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.307476 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.307482 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.307488 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.307494 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.307500 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.307506 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.307512 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.307518 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.307524 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.307568 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.307575 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.307581 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.307587 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.307593 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307599 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.307605 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.307611 | controller | --decode\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307617 | controller | | + public_secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:20 2026-01-21 11:38:24.307623 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.307629 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.307638 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.307644 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.307650 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.307656 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.307665 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.307671 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.307677 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.307683 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.307689 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.307694 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.307700 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.307706 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.307740 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.307746 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.307752 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.307758 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.307764 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.307770 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.307776 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.307782 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307796 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.307802 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.307808 | controller | 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.307814 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.307820 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.307826 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.307832 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.307842 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.307848 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.307858 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.307864 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.307869 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.307875 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.307881 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.307887 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.307893 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.307899 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.307905 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.307911 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.307917 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.307923 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.307931 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.307948 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:20 2026-01-21 11:38:24.307959 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.307967 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.307975 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.307982 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.307990 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.307998 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.308019 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.308027 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.308034 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.308048 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.308060 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.308069 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.308076 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.308084 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.308091 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.308099 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.308107 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.308114 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.308121 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.308128 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.308135 | controller | \ logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.308143 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308151 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.308158 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert='-----BEGIN 2026-01-21 11:38:24.308167 | controller | CERTIFICATE-----\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308174 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.308184 | controller | 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.308192 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.308199 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.308207 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.308215 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.308221 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.308227 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.308235 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.308243 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.308256 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.308281 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.308289 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.308296 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.308304 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.308312 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.308319 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.308327 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.308335 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.308342 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.308354 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.308361 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308369 | controller | | ++ base64 --decode\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308377 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.308384 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert='-----BEGIN 2026-01-21 11:38:24.308392 | controller | CERTIFICATE-----\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308399 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.308407 | controller | 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.308414 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.308422 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.308429 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.308437 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.308444 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.308452 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.308459 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.308471 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.308479 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.308492 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.308505 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.308513 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.308521 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.308545 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.308563 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.308579 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.308587 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.308599 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.308614 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.308622 | controller | CERTIFICATE-----'\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308630 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.308637 | controller | | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.308645 | controller | 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.308653 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.308660 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.308668 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.308674 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.308681 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.308689 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.308696 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.308704 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.308753 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.308773 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.308781 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.308788 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.308800 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.308815 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.308823 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.308831 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.308842 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.308858 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.308866 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.308874 | controller | CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:21 2026-01-21 11:38:24.308896 | controller | | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.308915 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw\n 2026-01-21 11:38:24.308926 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR\n 2026-01-21 11:38:24.308941 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0\n 2026-01-21 11:38:24.308948 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620\n 2026-01-21 11:38:24.308956 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5\n 2026-01-21 11:38:24.308963 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO\n 2026-01-21 11:38:24.308971 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9\n 2026-01-21 11:38:24.308978 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.308986 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.308993 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.309001 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX\n 2026-01-21 11:38:24.309014 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO\n 2026-01-21 11:38:24.309021 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO\n 2026-01-21 11:38:24.309029 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue\n 2026-01-21 11:38:24.309036 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2\n 2026-01-21 11:38:24.309044 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN\n 2026-01-21 11:38:24.309051 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz\n 2026-01-21 11:38:24.309059 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx\n 2026-01-21 11:38:24.309066 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ\n 2026-01-21 11:38:24.309079 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.309087 | controller | CERTIFICATE-----' ']'\n logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse 2026-01-21 11:38:24.309095 | controller | | test step completed 1-deploy-with-tlse\n logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309102 | controller | | starting test step 2-change-public-svc-certificate\n logger.go:42: 11:23:21 2026-01-21 11:38:24.309110 | controller | | watcher-tls-certs-change/2-change-public-svc-certificate | running command: [sh 2026-01-21 11:38:24.309117 | controller | -c oc patch Certificate -n $NAMESPACE watcher-public-svc --type='json' -p='[{\"op\": 2026-01-21 11:38:24.309125 | controller | \"replace\", \"path\": \"/spec/dnsNames\", \"value\":['watcher-public.watcher-kuttl-default.svc', 2026-01-21 11:38:24.309132 | controller | 'watcher-public.watcher-kuttl-default.svc.cluster.local']}]'\n ]\n logger.go:42: 2026-01-21 11:38:24.309140 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | certificate.cert-manager.io/watcher-public-svc 2026-01-21 11:38:24.309147 | controller | patched\n logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309155 | controller | | running command: [sh -c set -euxo pipefail\n svc_cert=$(oc rsh -n $NAMESPACE 2026-01-21 11:38:24.309163 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt)\n secret_cert=$(oc 2026-01-21 11:38:24.309184 | controller | get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.309193 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.309201 | controller | \ if [ \"${svc_cert}\" != \"${secret_cert}\" ]; then\n exit 1\n fi\n 2026-01-21 11:38:24.309208 | controller | \ ]\n logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309216 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt\n 2026-01-21 11:38:24.309223 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309233 | controller | | + svc_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309241 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.309249 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.309264 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309273 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n logger.go:42: 2026-01-21 11:38:24.309290 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.309303 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309311 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n logger.go:42: 2026-01-21 11:38:24.309319 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.309327 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309334 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n logger.go:42: 2026-01-21 11:38:24.309341 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.309349 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309356 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n logger.go:42: 2026-01-21 11:38:24.309363 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.309370 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309381 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n logger.go:42: 2026-01-21 11:38:24.309403 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.309412 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309420 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n logger.go:42: 2026-01-21 11:38:24.309427 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.309435 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309442 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n logger.go:42: 2026-01-21 11:38:24.309450 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.309458 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309465 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n logger.go:42: 2026-01-21 11:38:24.309473 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.309484 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309493 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n logger.go:42: 2026-01-21 11:38:24.309500 | controller | 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.309508 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309515 | controller | | -----END CERTIFICATE-----'\n logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309544 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.309553 | controller | \ logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309565 | controller | | ++ base64 --decode\n logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309572 | controller | | + secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309580 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.309587 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.309595 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309602 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n logger.go:42: 2026-01-21 11:38:24.309610 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.309617 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309625 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n logger.go:42: 2026-01-21 11:38:24.309632 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.309639 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309647 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n logger.go:42: 2026-01-21 11:38:24.309654 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.309663 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309670 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n logger.go:42: 2026-01-21 11:38:24.309678 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.309685 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309692 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n logger.go:42: 2026-01-21 11:38:24.309699 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.309706 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309734 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n logger.go:42: 2026-01-21 11:38:24.309742 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.309748 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309754 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n logger.go:42: 2026-01-21 11:38:24.309760 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.309766 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309772 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n logger.go:42: 2026-01-21 11:38:24.309778 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.309790 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309797 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n logger.go:42: 2026-01-21 11:38:24.309802 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.309808 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309814 | controller | | -----END CERTIFICATE-----'\n logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309820 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309826 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n logger.go:42: 2026-01-21 11:38:24.309844 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n 2026-01-21 11:38:24.309851 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309857 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n logger.go:42: 2026-01-21 11:38:24.309865 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n 2026-01-21 11:38:24.309875 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309892 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n logger.go:42: 2026-01-21 11:38:24.309900 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n 2026-01-21 11:38:24.309908 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309915 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n logger.go:42: 2026-01-21 11:38:24.309923 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n 2026-01-21 11:38:24.309935 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309944 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n logger.go:42: 2026-01-21 11:38:24.309952 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n 2026-01-21 11:38:24.309960 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309967 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n logger.go:42: 2026-01-21 11:38:24.309974 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n 2026-01-21 11:38:24.309982 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.309989 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n logger.go:42: 2026-01-21 11:38:24.310000 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n 2026-01-21 11:38:24.310007 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310014 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n logger.go:42: 2026-01-21 11:38:24.310021 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n 2026-01-21 11:38:24.310031 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310037 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n logger.go:42: 2026-01-21 11:38:24.310043 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n 2026-01-21 11:38:24.310049 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310055 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n logger.go:42: 2026-01-21 11:38:24.310060 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n 2026-01-21 11:38:24.310066 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310072 | controller | | -----END CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 2026-01-21 11:38:24.310079 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw\n 2026-01-21 11:38:24.310086 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310092 | controller | | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx\n logger.go:42: 2026-01-21 11:38:24.310098 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH\n 2026-01-21 11:38:24.310103 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310109 | controller | | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT\n logger.go:42: 2026-01-21 11:38:24.310115 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt\n 2026-01-21 11:38:24.310121 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310140 | controller | | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+\n logger.go:42: 2026-01-21 11:38:24.310148 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV\n 2026-01-21 11:38:24.310155 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310160 | controller | | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH\n logger.go:42: 2026-01-21 11:38:24.310167 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\n 2026-01-21 11:38:24.310172 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310178 | controller | | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK\n logger.go:42: 2026-01-21 11:38:24.310184 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl\n 2026-01-21 11:38:24.310190 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310196 | controller | | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG\n logger.go:42: 2026-01-21 11:38:24.310202 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG\n 2026-01-21 11:38:24.310208 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310214 | controller | | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q\n logger.go:42: 2026-01-21 11:38:24.310221 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN\n 2026-01-21 11:38:24.310269 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310277 | controller | | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV\n logger.go:42: 2026-01-21 11:38:24.310283 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN\n 2026-01-21 11:38:24.310289 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310301 | controller | | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck\n logger.go:42: 2026-01-21 11:38:24.310307 | controller | 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+\n 2026-01-21 11:38:24.310314 | controller | \ logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310322 | controller | | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R\n logger.go:42: 11:23:22 2026-01-21 11:38:24.310327 | controller | | watcher-tls-certs-change/2-change-public-svc-certificate | -----END CERTIFICATE-----' 2026-01-21 11:38:24.310333 | controller | ']'\n logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.310339 | controller | | test step completed 2-change-public-svc-certificate\n logger.go:42: 11:23:22 2026-01-21 11:38:24.310345 | controller | | watcher-tls-certs-change/3-change-internal-svc-certificate | starting test step 2026-01-21 11:38:24.310351 | controller | 3-change-internal-svc-certificate\n logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310357 | controller | | running command: [sh -c oc patch Certificate -n $NAMESPACE watcher-internal-svc 2026-01-21 11:38:24.310363 | controller | --type='json' -p='[{\"op\": \"replace\", \"path\": \"/spec/dnsNames\", \"value\":['watcher-internal.watcher-kuttl-default.svc', 2026-01-21 11:38:24.310369 | controller | 'watcher-internal.watcher-kuttl-default.svc.cluster.local']}]'\n ]\n logger.go:42: 2026-01-21 11:38:24.310375 | controller | 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate | certificate.cert-manager.io/watcher-internal-svc 2026-01-21 11:38:24.310381 | controller | patched\n logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310386 | controller | | running command: [sh -c set -euxo pipefail\n svc_cert=$(oc rsh -n $NAMESPACE 2026-01-21 11:38:24.310392 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n secret_cert=$(oc 2026-01-21 11:38:24.310398 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.310404 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.310410 | controller | \ if [ \"${svc_cert}\" != \"${secret_cert}\" ]; then\n exit 1\n fi\n 2026-01-21 11:38:24.310416 | controller | \ ]\n logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310422 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.310439 | controller | \ logger.go:42: 11:23:24 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310447 | controller | | error: Internal error occurred: error executing command in container: container 2026-01-21 11:38:24.310453 | controller | is not created or running\n logger.go:42: 11:23:24 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310458 | controller | | + svc_cert=\n logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310464 | controller | | running command: [sh -c set -euxo pipefail\n svc_cert=$(oc rsh -n $NAMESPACE 2026-01-21 11:38:24.310471 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n secret_cert=$(oc 2026-01-21 11:38:24.310483 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.310489 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.310495 | controller | \ if [ \"${svc_cert}\" != \"${secret_cert}\" ]; then\n exit 1\n fi\n 2026-01-21 11:38:24.310501 | controller | \ ]\n logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310507 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.310512 | controller | \ logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310518 | controller | | error: cannot exec into a container in a completed pod; current phase is Failed\n 2026-01-21 11:38:24.310524 | controller | \ logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310533 | controller | | + svc_cert=\n logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310539 | controller | | running command: [sh -c set -euxo pipefail\n svc_cert=$(oc rsh -n $NAMESPACE 2026-01-21 11:38:24.310545 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n secret_cert=$(oc 2026-01-21 11:38:24.310584 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.310590 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.310596 | controller | \ if [ \"${svc_cert}\" != \"${secret_cert}\" ]; then\n exit 1\n fi\n 2026-01-21 11:38:24.310603 | controller | \ ]\n logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310610 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.310616 | controller | \ logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310622 | controller | | error: unable to upgrade connection: container not found (\"watcher-api\")\n logger.go:42: 2026-01-21 11:38:24.310628 | controller | 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate | + svc_cert=\n 2026-01-21 11:38:24.310634 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310640 | controller | | running command: [sh -c set -euxo pipefail\n svc_cert=$(oc rsh -n $NAMESPACE 2026-01-21 11:38:24.310646 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt)\n secret_cert=$(oc 2026-01-21 11:38:24.310652 | controller | get -n $NAMESPACE secret cert-watcher-internal-svc -o jsonpath='{.data.tls\\.crt}' 2026-01-21 11:38:24.310657 | controller | | base64 --decode)\n # ensure that the svc secret and cert secret match\n 2026-01-21 11:38:24.310663 | controller | \ if [ \"${svc_cert}\" != \"${secret_cert}\" ]; then\n exit 1\n fi\n 2026-01-21 11:38:24.310669 | controller | \ ]\n logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310675 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt\n 2026-01-21 11:38:24.310681 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310687 | controller | | + svc_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310693 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.310699 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw\n 2026-01-21 11:38:24.310705 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310731 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF\n logger.go:42: 2026-01-21 11:38:24.310741 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z\n 2026-01-21 11:38:24.310764 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310771 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn\n logger.go:42: 2026-01-21 11:38:24.310777 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P\n 2026-01-21 11:38:24.310783 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310789 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58\n logger.go:42: 2026-01-21 11:38:24.310795 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV\n 2026-01-21 11:38:24.310801 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310807 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n logger.go:42: 2026-01-21 11:38:24.310813 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.310822 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310828 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0\n logger.go:42: 2026-01-21 11:38:24.310834 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy\n 2026-01-21 11:38:24.310840 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310846 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD\n logger.go:42: 2026-01-21 11:38:24.310852 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34\n 2026-01-21 11:38:24.310858 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310864 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w\n logger.go:42: 2026-01-21 11:38:24.310870 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68\n 2026-01-21 11:38:24.310876 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310882 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K\n logger.go:42: 2026-01-21 11:38:24.310890 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG\n 2026-01-21 11:38:24.310895 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310901 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2\n logger.go:42: 2026-01-21 11:38:24.310907 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF\n 2026-01-21 11:38:24.310913 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310919 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY\n logger.go:42: 2026-01-21 11:38:24.310925 | controller | 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate | P464iw==\n 2026-01-21 11:38:24.310931 | controller | \ logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310940 | controller | | -----END CERTIFICATE-----'\n logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310946 | controller | | ++ base64 --decode\n logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310952 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o 'jsonpath={.data.tls\\.crt}'\n 2026-01-21 11:38:24.310957 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310963 | controller | | + secret_cert='-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310969 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.310975 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw\n 2026-01-21 11:38:24.310981 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.310987 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF\n logger.go:42: 2026-01-21 11:38:24.311003 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z\n 2026-01-21 11:38:24.311009 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311016 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn\n logger.go:42: 2026-01-21 11:38:24.311023 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P\n 2026-01-21 11:38:24.311029 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311035 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58\n logger.go:42: 2026-01-21 11:38:24.311041 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV\n 2026-01-21 11:38:24.311047 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311053 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n logger.go:42: 2026-01-21 11:38:24.311059 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.311065 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311074 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0\n logger.go:42: 2026-01-21 11:38:24.311080 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy\n 2026-01-21 11:38:24.311086 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311092 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD\n logger.go:42: 2026-01-21 11:38:24.311098 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34\n 2026-01-21 11:38:24.311104 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311110 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w\n logger.go:42: 2026-01-21 11:38:24.311116 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68\n 2026-01-21 11:38:24.311122 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311131 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K\n logger.go:42: 2026-01-21 11:38:24.311137 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG\n 2026-01-21 11:38:24.311143 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311149 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2\n logger.go:42: 2026-01-21 11:38:24.311156 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF\n 2026-01-21 11:38:24.311163 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311169 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY\n logger.go:42: 2026-01-21 11:38:24.311176 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | P464iw==\n 2026-01-21 11:38:24.311183 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311189 | controller | | -----END CERTIFICATE-----'\n logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311195 | controller | | + '[' '-----BEGIN CERTIFICATE-----\n logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311201 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa\n logger.go:42: 2026-01-21 11:38:24.311207 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw\n 2026-01-21 11:38:24.311213 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311219 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF\n logger.go:42: 2026-01-21 11:38:24.311224 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z\n 2026-01-21 11:38:24.311231 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311240 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn\n logger.go:42: 2026-01-21 11:38:24.311266 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P\n 2026-01-21 11:38:24.311276 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311283 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58\n logger.go:42: 2026-01-21 11:38:24.311290 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV\n 2026-01-21 11:38:24.311298 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311305 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n logger.go:42: 2026-01-21 11:38:24.311313 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n 2026-01-21 11:38:24.311320 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311328 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0\n logger.go:42: 2026-01-21 11:38:24.311336 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy\n 2026-01-21 11:38:24.311349 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311361 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD\n logger.go:42: 2026-01-21 11:38:24.311368 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34\n 2026-01-21 11:38:24.311375 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311381 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w\n logger.go:42: 2026-01-21 11:38:24.311387 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68\n 2026-01-21 11:38:24.311393 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311399 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K\n logger.go:42: 2026-01-21 11:38:24.311405 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG\n 2026-01-21 11:38:24.311411 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311420 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2\n logger.go:42: 2026-01-21 11:38:24.311427 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF\n 2026-01-21 11:38:24.311434 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311440 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY\n logger.go:42: 2026-01-21 11:38:24.311446 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | P464iw==\n 2026-01-21 11:38:24.311452 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311458 | controller | | -----END CERTIFICATE-----' '!=' '-----BEGIN CERTIFICATE-----\n logger.go:42: 2026-01-21 11:38:24.311463 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa\n 2026-01-21 11:38:24.311469 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311475 | controller | | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw\n logger.go:42: 2026-01-21 11:38:24.311481 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF\n 2026-01-21 11:38:24.311487 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311493 | controller | | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z\n logger.go:42: 2026-01-21 11:38:24.311498 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn\n 2026-01-21 11:38:24.311504 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311510 | controller | | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P\n logger.go:42: 2026-01-21 11:38:24.311516 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58\n 2026-01-21 11:38:24.311522 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311542 | controller | | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV\n logger.go:42: 2026-01-21 11:38:24.311549 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE\n 2026-01-21 11:38:24.311555 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311567 | controller | | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG\n logger.go:42: 2026-01-21 11:38:24.311574 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0\n 2026-01-21 11:38:24.311580 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311586 | controller | | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy\n logger.go:42: 2026-01-21 11:38:24.311592 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD\n 2026-01-21 11:38:24.311598 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311604 | controller | | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34\n logger.go:42: 2026-01-21 11:38:24.311610 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w\n 2026-01-21 11:38:24.311616 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311621 | controller | | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68\n logger.go:42: 2026-01-21 11:38:24.311627 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K\n 2026-01-21 11:38:24.311633 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311639 | controller | | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG\n logger.go:42: 2026-01-21 11:38:24.311645 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2\n 2026-01-21 11:38:24.311651 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311656 | controller | | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF\n logger.go:42: 2026-01-21 11:38:24.311662 | controller | 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY\n 2026-01-21 11:38:24.311668 | controller | \ logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311674 | controller | | P464iw==\n logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311680 | controller | | -----END CERTIFICATE-----' ']'\n logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.311686 | controller | | test step completed 3-change-internal-svc-certificate\n logger.go:42: 11:23:28 2026-01-21 11:38:24.311692 | controller | | watcher-tls-certs-change/4-cleanup-watcher | starting test step 4-cleanup-watcher\n 2026-01-21 11:38:24.311699 | controller | \ logger.go:42: 11:23:38 | watcher-tls-certs-change/4-cleanup-watcher | test step 2026-01-21 11:38:24.311705 | controller | completed 4-cleanup-watcher\n logger.go:42: 11:23:38 | watcher-tls-certs-change/5-clenaup-certs 2026-01-21 11:38:24.311743 | controller | | starting test step 5-clenaup-certs\n logger.go:42: 11:23:38 | watcher-tls-certs-change/5-clenaup-certs 2026-01-21 11:38:24.311753 | controller | | test step completed 5-clenaup-certs\n logger.go:42: 11:23:38 | watcher-tls-certs-change 2026-01-21 11:38:24.311759 | controller | | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-tls\n logger.go:42: 2026-01-21 11:38:24.311765 | controller | 11:23:38 | watcher-tls | Skipping creation of user-supplied namespace: watcher-kuttl-default\n 2026-01-21 11:38:24.311772 | controller | \ logger.go:42: 11:23:38 | watcher-tls/0-cleanup-watcher | starting test step 2026-01-21 11:38:24.311789 | controller | 0-cleanup-watcher\n logger.go:42: 11:23:38 | watcher-tls/0-cleanup-watcher | 2026-01-21 11:38:24.311808 | controller | test step completed 0-cleanup-watcher\n logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.311815 | controller | | starting test step 1-deploy-with-tlse\n logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.311822 | controller | | Certificate:watcher-kuttl-default/watcher-internal-svc created\n logger.go:42: 2026-01-21 11:38:24.311829 | controller | 11:23:38 | watcher-tls/1-deploy-with-tlse | Certificate:watcher-kuttl-default/watcher-public-svc 2026-01-21 11:38:24.311836 | controller | created\n logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.311859 | controller | created\n logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.311868 | controller | [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient -- openstack 2026-01-21 11:38:24.311875 | controller | service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc 2026-01-21 11:38:24.311883 | controller | exec -n ${NAMESPACE} openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.311890 | controller | -c Type -c ID | grep watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.311898 | controller | keystoneservice watcher -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ 2026-01-21 11:38:24.311905 | controller | -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" 2026-01-21 11:38:24.311912 | controller | ]\n [ \"$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o 2026-01-21 11:38:24.311919 | controller | jsonpath='{.data.my\\.cnf}'|base64 -d|grep -c 'ssl=1')\" == 1 ]\n [ \"$(oc 2026-01-21 11:38:24.311926 | controller | get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath='{.data.00-default\\.conf}'|base64 2026-01-21 11:38:24.311933 | controller | -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" == 2 2026-01-21 11:38:24.311940 | controller | ]\n # check that both endpoints have https set\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.311947 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.311954 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.311962 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.311969 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.311977 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.311985 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.311992 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.311998 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.312005 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.312012 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.312019 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312027 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.312034 | controller | value -c Name -c Type\n logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312041 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:23:44 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312048 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:23:45 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312056 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.312068 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.312076 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.312083 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.312090 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.312097 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.312104 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n ${NAMESPACE} secret 2026-01-21 11:38:24.312111 | controller | watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep -c 'ssl=1')\" 2026-01-21 11:38:24.312119 | controller | == 1 ]\n [ \"$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.312127 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.312134 | controller | == 2 ]\n # check that both endpoints have https set\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.312141 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.312157 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.312164 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.312171 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.312179 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.312186 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.312208 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.312217 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.312225 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.312237 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.312245 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:23:45 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312253 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.312260 | controller | value -c Name -c Type\n logger.go:42: 11:23:45 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312267 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:23:48 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312275 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:23:49 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312283 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.312291 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.312299 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.312306 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.312314 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.312322 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.312329 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n ${NAMESPACE} secret 2026-01-21 11:38:24.312336 | controller | watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep -c 'ssl=1')\" 2026-01-21 11:38:24.312344 | controller | == 1 ]\n [ \"$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.312356 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.312363 | controller | == 2 ]\n # check that both endpoints have https set\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.312370 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.312377 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.312385 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.312392 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.312399 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.312407 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.312414 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.312423 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.312431 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.312438 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.312445 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:23:49 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312452 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.312459 | controller | value -c Name -c Type\n logger.go:42: 11:23:49 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312472 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312481 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312488 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.312495 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312503 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312510 | controller | | ++ grep watcher\n logger.go:42: 11:23:57 | watcher-tls/1-deploy-with-tlse | 2026-01-21 11:38:24.312518 | controller | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:23:57 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312526 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.312534 | controller | \ logger.go:42: 11:23:58 | watcher-tls/1-deploy-with-tlse | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.312541 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:23:58 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312570 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.312580 | controller | \ logger.go:42: 11:23:58 | watcher-tls/1-deploy-with-tlse | + '[' -n '' ']'\n 2026-01-21 11:38:24.312587 | controller | \ logger.go:42: 11:23:59 | watcher-tls/1-deploy-with-tlse | running command: [sh 2026-01-21 11:38:24.312595 | controller | -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient -- openstack 2026-01-21 11:38:24.312602 | controller | service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc 2026-01-21 11:38:24.312609 | controller | exec -n ${NAMESPACE} openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.312616 | controller | -c Type -c ID | grep watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.312623 | controller | keystoneservice watcher -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ 2026-01-21 11:38:24.312636 | controller | -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" 2026-01-21 11:38:24.312644 | controller | ]\n [ \"$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o 2026-01-21 11:38:24.312651 | controller | jsonpath='{.data.my\\.cnf}'|base64 -d|grep -c 'ssl=1')\" == 1 ]\n [ \"$(oc 2026-01-21 11:38:24.312659 | controller | get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath='{.data.00-default\\.conf}'|base64 2026-01-21 11:38:24.312666 | controller | -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" == 2 2026-01-21 11:38:24.312674 | controller | ]\n # check that both endpoints have https set\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.312681 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.312688 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.312696 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.312702 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.312732 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.312745 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.312751 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.312758 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.312775 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.312785 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.312793 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:23:59 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312801 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.312808 | controller | value -c Name -c Type\n logger.go:42: 11:23:59 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312815 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312823 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312830 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.312838 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312845 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312853 | controller | | ++ grep watcher\n logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | 2026-01-21 11:38:24.312860 | controller | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312868 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.312875 | controller | \ logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.312883 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.312890 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.312918 | controller | \ logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.312935 | controller | ']'\n logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ oc get -n 2026-01-21 11:38:24.312949 | controller | watcher-kuttl-default secret watcher-kuttl-api-config-data -o 'jsonpath={.data.my\\.cnf}'\n 2026-01-21 11:38:24.312957 | controller | \ logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ grep -c ssl=1\n 2026-01-21 11:38:24.312964 | controller | \ logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.312990 | controller | 11:24:06 | watcher-tls/1-deploy-with-tlse | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.313001 | controller | 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default secret 2026-01-21 11:38:24.313018 | controller | watcher-kuttl-api-config-data -o 'jsonpath={.data.00-default\\.conf}'\n logger.go:42: 2026-01-21 11:38:24.313026 | controller | 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'\n 2026-01-21 11:38:24.313034 | controller | \ logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.313042 | controller | 11:24:06 | watcher-tls/1-deploy-with-tlse | + '[' 2 == 2 ']'\n logger.go:42: 2026-01-21 11:38:24.313056 | controller | 11:24:06 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.313066 | controller | -- openstack endpoint list\n logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313073 | controller | | + grep infra-optim\n logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313081 | controller | | ++ grep -c https\n logger.go:42: 11:24:09 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313093 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:24:10 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313106 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.313113 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.313120 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.313127 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.313134 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.313141 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.313149 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n ${NAMESPACE} secret 2026-01-21 11:38:24.313156 | controller | watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep -c 'ssl=1')\" 2026-01-21 11:38:24.313164 | controller | == 1 ]\n [ \"$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.313171 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.313178 | controller | == 2 ]\n # check that both endpoints have https set\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.313186 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.313193 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.313201 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.313213 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.313221 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.313229 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.313244 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.313260 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.313269 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.313283 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.313291 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:24:10 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313298 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.313306 | controller | value -c Name -c Type\n logger.go:42: 11:24:10 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313314 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313322 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313329 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.313336 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313344 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313351 | controller | | ++ grep watcher\n logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | 2026-01-21 11:38:24.313358 | controller | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313365 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.313389 | controller | \ logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.313399 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313406 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.313414 | controller | \ logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.313422 | controller | ']'\n logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ base64 -d\n 2026-01-21 11:38:24.313429 | controller | \ logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ grep -c ssl=1\n 2026-01-21 11:38:24.313437 | controller | \ logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.313445 | controller | secret watcher-kuttl-api-config-data -o 'jsonpath={.data.my\\.cnf}'\n logger.go:42: 2026-01-21 11:38:24.313453 | controller | 11:24:16 | watcher-tls/1-deploy-with-tlse | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.313467 | controller | 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default secret 2026-01-21 11:38:24.313479 | controller | watcher-kuttl-api-config-data -o 'jsonpath={.data.00-default\\.conf}'\n logger.go:42: 2026-01-21 11:38:24.313486 | controller | 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'\n 2026-01-21 11:38:24.313494 | controller | \ logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.313501 | controller | 11:24:17 | watcher-tls/1-deploy-with-tlse | + '[' 2 == 2 ']'\n logger.go:42: 2026-01-21 11:38:24.313508 | controller | 11:24:17 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.313522 | controller | -- openstack endpoint list\n logger.go:42: 11:24:17 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313530 | controller | | + grep infra-optim\n logger.go:42: 11:24:17 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313538 | controller | | ++ grep -c https\n logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313546 | controller | | + '[' 2 == 2 ']'\n logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313554 | controller | | ++ oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n 2026-01-21 11:38:24.313566 | controller | \ logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse | + '[' '' == '' ']'\n 2026-01-21 11:38:24.313574 | controller | \ logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse | + exit 0\n logger.go:42: 2026-01-21 11:38:24.313581 | controller | 11:24:20 | watcher-tls/1-deploy-with-tlse | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.313598 | controller | \ oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.313607 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.313614 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID | grep 2026-01-21 11:38:24.313622 | controller | watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} keystoneservice watcher 2026-01-21 11:38:24.313629 | controller | -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.313636 | controller | watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get 2026-01-21 11:38:24.313643 | controller | -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 2026-01-21 11:38:24.313650 | controller | -d|grep -c 'ssl=1')\" == 1 ]\n [ \"$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.313657 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.313665 | controller | == 2 ]\n # check that both endpoints have https set\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.313672 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.313680 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.313687 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.313694 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.313702 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.313731 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.313744 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.313752 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.313759 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.313783 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.313791 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:24:20 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313799 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.313806 | controller | value -c Name -c Type\n logger.go:42: 11:24:20 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313813 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313821 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313829 | controller | | ++ grep watcher\n logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse | 2026-01-21 11:38:24.313836 | controller | ++ awk '{print $1}'\n logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313843 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.313850 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313857 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313871 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.313885 | controller | \ logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.313893 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.313900 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.313907 | controller | \ logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.313914 | controller | ']'\n logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ base64 -d\n 2026-01-21 11:38:24.313921 | controller | \ logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ grep -c ssl=1\n 2026-01-21 11:38:24.313928 | controller | \ logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.313936 | controller | secret watcher-kuttl-api-config-data -o 'jsonpath={.data.my\\.cnf}'\n logger.go:42: 2026-01-21 11:38:24.313943 | controller | 11:24:26 | watcher-tls/1-deploy-with-tlse | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.313950 | controller | 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default secret 2026-01-21 11:38:24.313965 | controller | watcher-kuttl-api-config-data -o 'jsonpath={.data.00-default\\.conf}'\n logger.go:42: 2026-01-21 11:38:24.313972 | controller | 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'\n 2026-01-21 11:38:24.313980 | controller | \ logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.313987 | controller | 11:24:26 | watcher-tls/1-deploy-with-tlse | + '[' 2 == 2 ']'\n logger.go:42: 2026-01-21 11:38:24.313994 | controller | 11:24:26 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.314001 | controller | -- openstack endpoint list\n logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.314008 | controller | | + grep infra-optim\n logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.314015 | controller | | ++ grep -c https\n logger.go:42: 11:24:28 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.314022 | controller | | + '[' 2 == 2 ']'\n logger.go:42: 11:24:28 | watcher-tls/1-deploy-with-tlse 2026-01-21 11:38:24.314029 | controller | | ++ oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n 2026-01-21 11:38:24.314037 | controller | \ logger.go:42: 11:24:29 | watcher-tls/1-deploy-with-tlse | + '[' '' == '' ']'\n 2026-01-21 11:38:24.314045 | controller | \ logger.go:42: 11:24:29 | watcher-tls/1-deploy-with-tlse | + exit 0\n logger.go:42: 2026-01-21 11:38:24.314052 | controller | 11:24:29 | watcher-tls/1-deploy-with-tlse | test step completed 1-deploy-with-tlse\n 2026-01-21 11:38:24.314059 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | starting test step 2-patch-mtls\n 2026-01-21 11:38:24.314067 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.314074 | controller | set -euxo pipefail\n oc patch oscp -n $NAMESPACE openstack --type='json' 2026-01-21 11:38:24.314081 | controller | -p='[{\"op\": \"replace\", \"path\": \"/spec/memcached/templates/memcached/tls/mtls/sslVerifyMode\", 2026-01-21 11:38:24.314088 | controller | \"value\": \"Request\"}]'\n ]\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314096 | controller | | + oc patch oscp -n watcher-kuttl-default openstack --type=json '-p=[{\"op\": \"replace\", 2026-01-21 11:38:24.314103 | controller | \"path\": \"/spec/memcached/templates/memcached/tls/mtls/sslVerifyMode\", \"value\": 2026-01-21 11:38:24.314122 | controller | \"Request\"}]'\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | openstackcontrolplane.core.openstack.org/openstack 2026-01-21 11:38:24.314134 | controller | patched\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | running command: 2026-01-21 11:38:24.314142 | controller | [sh -c set -euxo pipefail\n \n oc project ${NAMESPACE}\n # 2026-01-21 11:38:24.314149 | controller | Get pod names for each watcher service\n APIPOD=$(oc get pods -l service=watcher-api 2026-01-21 11:38:24.314156 | controller | -o jsonpath='{.items[0].metadata.name}')\n APPLIERPOD=$(oc get pods -l service=watcher-applier 2026-01-21 11:38:24.314164 | controller | -o jsonpath='{.items[0].metadata.name}')\n DECISIONENGINEPOD=$(oc get pods 2026-01-21 11:38:24.314171 | controller | -l service=watcher-decision-engine -o jsonpath='{.items[0].metadata.name}')\n \n 2026-01-21 11:38:24.314179 | controller | \ # Verify memcached mTLS config parameters in watcher-api config\n if 2026-01-21 11:38:24.314188 | controller | [ -n \"${APIPOD}\" ]; then\n echo \"Checking watcher-api config...\"\n 2026-01-21 11:38:24.314195 | controller | \ [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314202 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.314209 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314216 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.314224 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314231 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.314238 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314245 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n # Verify 2026-01-21 11:38:24.314252 | controller | mTLS config parameters in memcached backend config\n [ $(echo $(oc rsh 2026-01-21 11:38:24.314260 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.314267 | controller | \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.314274 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.314281 | controller | \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.314289 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.314300 | controller | \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n exit 2026-01-21 11:38:24.314308 | controller | 1\n fi\n \n # Verify memcached mTLS config parameters in watcher-applier 2026-01-21 11:38:24.314315 | controller | config\n if [ -n \"${APPLIERPOD}\" ]; then\n echo \"Checking watcher-applier 2026-01-21 11:38:24.314322 | controller | config...\"\n [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314329 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.314336 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314344 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.314351 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314358 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.314365 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314373 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.314385 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314393 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.314400 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314407 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.314414 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314422 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.314429 | controller | \ exit 1\n fi\n \n # Verify memcached mTLS config parameters 2026-01-21 11:38:24.314436 | controller | in watcher-decision-engine config\n if [ -n \"${DECISIONENGINEPOD}\" ]; then\n 2026-01-21 11:38:24.314443 | controller | \ echo \"Checking watcher-decision-engine config...\"\n [ $(echo 2026-01-21 11:38:24.314463 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314471 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.314478 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314485 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.314492 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314500 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.314507 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314514 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.314522 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314529 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.314536 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314544 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.314576 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.314586 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.314593 | controller | \ exit 1\n fi\n ]\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314601 | controller | | + oc project watcher-kuttl-default\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314608 | controller | | Now using project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.314615 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.314627 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314635 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314642 | controller | | ++ oc get pods -l service=watcher-applier -o 'jsonpath={.items[0].metadata.name}'\n 2026-01-21 11:38:24.314649 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0\n 2026-01-21 11:38:24.314657 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.314669 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314676 | controller | | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0\n logger.go:42: 11:24:29 2026-01-21 11:38:24.314687 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 2026-01-21 11:38:24.314694 | controller | 11:24:29 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-api config...'\n 2026-01-21 11:38:24.314702 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | Checking watcher-api config...\n 2026-01-21 11:38:24.314725 | controller | \ logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.314737 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.314744 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.314752 | controller | \ logger.go:42: 11:24:30 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.314760 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.314770 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.314777 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.314785 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.314793 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.314801 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.314808 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.314816 | controller | project_domain_name = Default project_name = service user_domain_name = Default 2026-01-21 11:38:24.314840 | controller | password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.314848 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.314856 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.314863 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.314870 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.314879 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.314886 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.314894 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.314901 | controller | tls_enabled=true memcache_dead_retry = 30 '[prometheus_client]' host = metric-storage-prometheus.watcher-kuttl-default.svc 2026-01-21 11:38:24.314914 | controller | port = 9090 cafile = /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' 2026-01-21 11:38:24.314921 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.314929 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.314936 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.314948 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.314955 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.314963 | controller | period = 900\n logger.go:42: 11:24:30 | watcher-tls/2-patch-mtls | + '[' 0 == 2026-01-21 11:38:24.314971 | controller | 1 ']'\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | running command: 2026-01-21 11:38:24.314978 | controller | [sh -c set -euxo pipefail\n \n oc project ${NAMESPACE}\n # 2026-01-21 11:38:24.314986 | controller | Get pod names for each watcher service\n APIPOD=$(oc get pods -l service=watcher-api 2026-01-21 11:38:24.314993 | controller | -o jsonpath='{.items[0].metadata.name}')\n APPLIERPOD=$(oc get pods -l service=watcher-applier 2026-01-21 11:38:24.315001 | controller | -o jsonpath='{.items[0].metadata.name}')\n DECISIONENGINEPOD=$(oc get pods 2026-01-21 11:38:24.315008 | controller | -l service=watcher-decision-engine -o jsonpath='{.items[0].metadata.name}')\n \n 2026-01-21 11:38:24.315016 | controller | \ # Verify memcached mTLS config parameters in watcher-api config\n if 2026-01-21 11:38:24.315023 | controller | [ -n \"${APIPOD}\" ]; then\n echo \"Checking watcher-api config...\"\n 2026-01-21 11:38:24.315030 | controller | \ [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315038 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.315049 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315064 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.315071 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315078 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.315085 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315092 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n # Verify 2026-01-21 11:38:24.315099 | controller | mTLS config parameters in memcached backend config\n [ $(echo $(oc rsh 2026-01-21 11:38:24.315107 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.315114 | controller | \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.315122 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.315129 | controller | \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.315136 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.315143 | controller | \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n exit 2026-01-21 11:38:24.315153 | controller | 1\n fi\n \n # Verify memcached mTLS config parameters in watcher-applier 2026-01-21 11:38:24.315170 | controller | config\n if [ -n \"${APPLIERPOD}\" ]; then\n echo \"Checking watcher-applier 2026-01-21 11:38:24.315178 | controller | config...\"\n [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315201 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.315211 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315218 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.315225 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315235 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.315248 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315256 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.315264 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315275 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.315283 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315289 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.315295 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315301 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.315307 | controller | \ exit 1\n fi\n \n # Verify memcached mTLS config parameters 2026-01-21 11:38:24.315313 | controller | in watcher-decision-engine config\n if [ -n \"${DECISIONENGINEPOD}\" ]; then\n 2026-01-21 11:38:24.315319 | controller | \ echo \"Checking watcher-decision-engine config...\"\n [ $(echo 2026-01-21 11:38:24.315324 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315330 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.315338 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315346 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.315354 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315361 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.315369 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315377 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.315384 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315392 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.315399 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315407 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.315414 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315422 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.315429 | controller | \ exit 1\n fi\n ]\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315437 | controller | | + oc project watcher-kuttl-default\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315444 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.315456 | controller | \ logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.315463 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315470 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315478 | controller | | ++ oc get pods -l service=watcher-applier -o 'jsonpath={.items[0].metadata.name}'\n 2026-01-21 11:38:24.315490 | controller | \ logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0\n 2026-01-21 11:38:24.315497 | controller | \ logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.315508 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315523 | controller | | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0\n logger.go:42: 11:24:31 2026-01-21 11:38:24.315531 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 2026-01-21 11:38:24.315556 | controller | 11:24:31 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-api config...'\n 2026-01-21 11:38:24.315567 | controller | \ logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | Checking watcher-api config...\n 2026-01-21 11:38:24.315574 | controller | \ logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.315581 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315588 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.315595 | controller | \ logger.go:42: 11:24:32 | watcher-tls/2-patch-mtls | error: Internal error occurred: 2026-01-21 11:38:24.315602 | controller | error executing command in container: container is not created or running\n logger.go:42: 2026-01-21 11:38:24.315609 | controller | 11:24:32 | watcher-tls/2-patch-mtls | ++ echo\n logger.go:42: 11:24:32 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.315617 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | running 2026-01-21 11:38:24.315624 | controller | command: [sh -c set -euxo pipefail\n \n oc project ${NAMESPACE}\n 2026-01-21 11:38:24.315631 | controller | \ # Get pod names for each watcher service\n APIPOD=$(oc get pods -l 2026-01-21 11:38:24.315639 | controller | service=watcher-api -o jsonpath='{.items[0].metadata.name}')\n APPLIERPOD=$(oc 2026-01-21 11:38:24.315646 | controller | get pods -l service=watcher-applier -o jsonpath='{.items[0].metadata.name}')\n DECISIONENGINEPOD=$(oc 2026-01-21 11:38:24.315654 | controller | get pods -l service=watcher-decision-engine -o jsonpath='{.items[0].metadata.name}')\n 2026-01-21 11:38:24.315661 | controller | \ \n # Verify memcached mTLS config parameters in watcher-api config\n 2026-01-21 11:38:24.315669 | controller | \ if [ -n \"${APIPOD}\" ]; then\n echo \"Checking watcher-api config...\"\n 2026-01-21 11:38:24.315677 | controller | \ [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315684 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.315692 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315699 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.315742 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315764 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.315772 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315780 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n # Verify 2026-01-21 11:38:24.315788 | controller | mTLS config parameters in memcached backend config\n [ $(echo $(oc rsh 2026-01-21 11:38:24.315796 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.315803 | controller | \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.315811 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.315823 | controller | \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.315831 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.315838 | controller | \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n exit 2026-01-21 11:38:24.315846 | controller | 1\n fi\n \n # Verify memcached mTLS config parameters in watcher-applier 2026-01-21 11:38:24.315853 | controller | config\n if [ -n \"${APPLIERPOD}\" ]; then\n echo \"Checking watcher-applier 2026-01-21 11:38:24.315861 | controller | config...\"\n [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315869 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.315876 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315884 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.315891 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315898 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.315905 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315912 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.315920 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315926 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.315947 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315953 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.315959 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.315967 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.315982 | controller | \ exit 1\n fi\n \n # Verify memcached mTLS config parameters 2026-01-21 11:38:24.315994 | controller | in watcher-decision-engine config\n if [ -n \"${DECISIONENGINEPOD}\" ]; then\n 2026-01-21 11:38:24.316002 | controller | \ echo \"Checking watcher-decision-engine config...\"\n [ $(echo 2026-01-21 11:38:24.316010 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316018 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.316028 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316044 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.316052 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316059 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.316067 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316075 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.316089 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316102 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.316111 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316128 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.316136 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316143 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.316150 | controller | \ exit 1\n fi\n ]\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316157 | controller | | + oc project watcher-kuttl-default\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316165 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.316172 | controller | \ logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.316180 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316187 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316197 | controller | | ++ oc get pods -l service=watcher-applier -o 'jsonpath={.items[0].metadata.name}'\n 2026-01-21 11:38:24.316204 | controller | \ logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0\n 2026-01-21 11:38:24.316210 | controller | \ logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.316217 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316224 | controller | | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0\n logger.go:42: 11:24:33 2026-01-21 11:38:24.316232 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 2026-01-21 11:38:24.316239 | controller | 11:24:33 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-api config...'\n 2026-01-21 11:38:24.316247 | controller | \ logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | Checking watcher-api config...\n 2026-01-21 11:38:24.316254 | controller | \ logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.316261 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316273 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.316281 | controller | \ logger.go:42: 11:24:34 | watcher-tls/2-patch-mtls | error: unable to upgrade 2026-01-21 11:38:24.316288 | controller | connection: container not found (\"watcher-api\")\n logger.go:42: 11:24:34 | 2026-01-21 11:38:24.316295 | controller | watcher-tls/2-patch-mtls | ++ echo\n logger.go:42: 11:24:34 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316303 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | running 2026-01-21 11:38:24.316326 | controller | command: [sh -c set -euxo pipefail\n \n oc project ${NAMESPACE}\n 2026-01-21 11:38:24.316335 | controller | \ # Get pod names for each watcher service\n APIPOD=$(oc get pods -l 2026-01-21 11:38:24.316343 | controller | service=watcher-api -o jsonpath='{.items[0].metadata.name}')\n APPLIERPOD=$(oc 2026-01-21 11:38:24.316350 | controller | get pods -l service=watcher-applier -o jsonpath='{.items[0].metadata.name}')\n DECISIONENGINEPOD=$(oc 2026-01-21 11:38:24.316358 | controller | get pods -l service=watcher-decision-engine -o jsonpath='{.items[0].metadata.name}')\n 2026-01-21 11:38:24.316366 | controller | \ \n # Verify memcached mTLS config parameters in watcher-api config\n 2026-01-21 11:38:24.316373 | controller | \ if [ -n \"${APIPOD}\" ]; then\n echo \"Checking watcher-api config...\"\n 2026-01-21 11:38:24.316380 | controller | \ [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316393 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.316401 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316408 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.316415 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316423 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.316430 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316437 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n # Verify 2026-01-21 11:38:24.316444 | controller | mTLS config parameters in memcached backend config\n [ $(echo $(oc rsh 2026-01-21 11:38:24.316452 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.316459 | controller | \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.316466 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.316473 | controller | \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.316481 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.316488 | controller | \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n exit 2026-01-21 11:38:24.316496 | controller | 1\n fi\n \n # Verify memcached mTLS config parameters in watcher-applier 2026-01-21 11:38:24.316503 | controller | config\n if [ -n \"${APPLIERPOD}\" ]; then\n echo \"Checking watcher-applier 2026-01-21 11:38:24.316511 | controller | config...\"\n [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316518 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.316525 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316533 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.316540 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316547 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.316554 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316567 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.316575 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316582 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.316590 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316596 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.316604 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316611 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.316618 | controller | \ exit 1\n fi\n \n # Verify memcached mTLS config parameters 2026-01-21 11:38:24.316626 | controller | in watcher-decision-engine config\n if [ -n \"${DECISIONENGINEPOD}\" ]; then\n 2026-01-21 11:38:24.316634 | controller | \ echo \"Checking watcher-decision-engine config...\"\n [ $(echo 2026-01-21 11:38:24.316641 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316653 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.316675 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316684 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.316691 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316698 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.316706 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316733 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.316748 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316757 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.316764 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316771 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.316779 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.316786 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.316794 | controller | \ exit 1\n fi\n ]\n logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316801 | controller | | + oc project watcher-kuttl-default\n logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316809 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.316816 | controller | \ logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.316823 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316831 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316838 | controller | | ++ oc get pods -l service=watcher-applier -o 'jsonpath={.items[0].metadata.name}'\n 2026-01-21 11:38:24.316846 | controller | \ logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0\n 2026-01-21 11:38:24.316853 | controller | \ logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.316861 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316869 | controller | | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0\n logger.go:42: 11:24:36 2026-01-21 11:38:24.316877 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 2026-01-21 11:38:24.316884 | controller | 11:24:36 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-api config...'\n 2026-01-21 11:38:24.316891 | controller | \ logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | Checking watcher-api config...\n 2026-01-21 11:38:24.316898 | controller | \ logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.316909 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.316923 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.316931 | controller | \ logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.316938 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.316952 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.316959 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.316966 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.316974 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.316981 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.316989 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.316997 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.317004 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.317011 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.317038 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.317057 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.317075 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.317093 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.317101 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.317109 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.317117 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.317124 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.317131 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.317138 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.317145 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.317152 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.317164 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.317171 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.317178 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.317185 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.317192 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.317198 | controller | 11:24:36 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:36 2026-01-21 11:38:24.317205 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_keyfile = /etc/pki/tls/private/mtls.key'\n 2026-01-21 11:38:24.317212 | controller | \ logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.317219 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.317233 | controller | 11:24:36 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.317240 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.317248 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.317255 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.317264 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.317271 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.317279 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.317287 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.317299 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.317306 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.317313 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.317320 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.317327 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.317334 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.317341 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.317348 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.317355 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.317365 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.317385 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.317393 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.317400 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.317407 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.317413 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.317420 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.317427 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.317434 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.317441 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.317448 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.317455 | controller | 11:24:36 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:36 2026-01-21 11:38:24.317462 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt'\n 2026-01-21 11:38:24.317474 | controller | \ logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.317481 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.317496 | controller | 11:24:36 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.317508 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.317516 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.317523 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.317531 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.317538 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.317546 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.317553 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.317560 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.317568 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.317575 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.317582 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.317591 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.317598 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.317605 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.317612 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.317619 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.317627 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.317634 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.317641 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.317648 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.317655 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.317662 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.317670 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.317677 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.317684 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.319231 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.319280 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.319291 | controller | 11:24:36 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:36 2026-01-21 11:38:24.319306 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_enabled = true'\n logger.go:42: 2026-01-21 11:38:24.319314 | controller | 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.319322 | controller | cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.319330 | controller | | ++ echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.319372 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.319381 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.319389 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.319397 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.319405 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.319412 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.319420 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.319428 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.319436 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.319444 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.319455 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.319464 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.319471 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.319479 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.319487 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.319495 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.319503 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.319510 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.319519 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.319527 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.319535 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.319543 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.319551 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.319559 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.319566 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.319574 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.319587 | controller | 11:24:37 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:37 2026-01-21 11:38:24.319595 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt\n 2026-01-21 11:38:24.319603 | controller | \ logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.319611 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.319619 | controller | 11:24:37 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.319641 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.319649 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.319657 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.319665 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.319673 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.319681 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.319688 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.319696 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.319704 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.319729 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.319753 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.319762 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.319770 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.319778 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.319785 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.319793 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.319801 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.319809 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.319817 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.319825 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.319833 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.319841 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.319848 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.319856 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.319864 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.319872 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.319885 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.319893 | controller | 11:24:37 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:37 2026-01-21 11:38:24.319901 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key\n 2026-01-21 11:38:24.319909 | controller | \ logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.319921 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.319929 | controller | 11:24:37 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.319937 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.319945 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.319952 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.319960 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.319967 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.319985 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.320005 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320040 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.320050 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.320058 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.320066 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320074 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.320082 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.320090 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320098 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.320111 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.320126 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320135 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.320150 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.320160 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.320168 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.320187 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.320198 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.320205 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.320220 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.320228 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.320236 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.320244 | controller | 11:24:37 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:37 2026-01-21 11:38:24.320251 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\n 2026-01-21 11:38:24.320259 | controller | \ logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.320267 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.320275 | controller | 11:24:37 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.320283 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.320291 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.320298 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.320306 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.320314 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.320322 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.320330 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320338 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.320345 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.320353 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.320361 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320369 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.320382 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.320390 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320415 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.320424 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.320432 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320440 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.320448 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.320456 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.320464 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.320471 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.320479 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.320491 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.320499 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.320506 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.320514 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.320522 | controller | 11:24:37 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:37 2026-01-21 11:38:24.320530 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-applier-0 ']'\n logger.go:42: 2026-01-21 11:38:24.320538 | controller | 11:24:37 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-applier config...'\n 2026-01-21 11:38:24.320546 | controller | \ logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | Checking watcher-applier 2026-01-21 11:38:24.320553 | controller | config...\n logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.320574 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.320586 | controller | | +++ oc rsh -c watcher-applier watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.320593 | controller | \ logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.320600 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.320607 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.320615 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.320640 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.320648 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.320655 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320663 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.320670 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.320686 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.320698 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320706 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.320732 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.320745 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320753 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.320761 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.320769 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320777 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.320807 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.320816 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.320824 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.320832 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.320840 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.320847 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.320855 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.320863 | controller | period = 900\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.320871 | controller | 1 ']'\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_keyfile 2026-01-21 11:38:24.320879 | controller | = /etc/pki/tls/private/mtls.key'\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.320886 | controller | | +++ oc rsh -c watcher-applier watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.320894 | controller | \ logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.320902 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.320910 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.320918 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.320926 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.320933 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.320941 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.320949 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.320957 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.320969 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.320982 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.320996 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321004 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.321011 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.321018 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321034 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.321052 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.321060 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.321068 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.321081 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.321088 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.321096 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.321104 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.321112 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.321119 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.321127 | controller | period = 900\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.321135 | controller | 1 ']'\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_cafile 2026-01-21 11:38:24.321143 | controller | = /etc/pki/tls/certs/mtls-ca.crt'\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.321169 | controller | | +++ oc rsh -c watcher-applier watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.321179 | controller | \ logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.321186 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.321194 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.321202 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.321213 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.321230 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.321240 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.321250 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.321259 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.321270 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.321279 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.321295 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321308 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.321316 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.321323 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321330 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.321338 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.321346 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.321354 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.321362 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.321382 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.321391 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.321399 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.321407 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.321415 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.321422 | controller | period = 900\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.321430 | controller | 1 ']'\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_enabled 2026-01-21 11:38:24.321438 | controller | = true'\n logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.321445 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.321453 | controller | 11:24:39 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.321459 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.321472 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.321486 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.321500 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.321508 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.321516 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.321549 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.321571 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.321583 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.321601 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.321610 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321618 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.321636 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.321646 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321653 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.321660 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.321668 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.321676 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.321690 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.321704 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.321759 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.321780 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.321793 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.321809 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.321818 | controller | period = 900\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.321826 | controller | 1 ']'\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt\n 2026-01-21 11:38:24.321839 | controller | \ logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.321847 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.321864 | controller | 11:24:39 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.321875 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.321883 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.321898 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.321911 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.321920 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.321929 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.321936 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.321944 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.321957 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.321971 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.321980 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.321988 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.322000 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.322022 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.322030 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.322056 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.322075 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.322085 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.322093 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.322104 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.322121 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.322144 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.322153 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.322162 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.322179 | controller | period = 900\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.322189 | controller | 1 ']'\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key\n 2026-01-21 11:38:24.322197 | controller | \ logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.322211 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.322224 | controller | 11:24:39 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.322233 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.322241 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.322251 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.322260 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.322268 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.322275 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.322282 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.322295 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.322303 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.322310 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.322318 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.322325 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.322332 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.322339 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.322347 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.322354 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.322362 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.322369 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.322376 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.322383 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.322391 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.322398 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.322411 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.322419 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.322427 | controller | period = 900\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.322452 | controller | 1 ']'\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\n 2026-01-21 11:38:24.322462 | controller | \ logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.322470 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.322477 | controller | 11:24:39 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.322484 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.322492 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.322499 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.322507 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.322514 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.322522 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.322530 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.322537 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.322545 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.322594 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.322603 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.322610 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.322618 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.322625 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.322632 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.322639 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.322648 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.322656 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.322664 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.322671 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.322678 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.322686 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.322693 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.322709 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.322742 | controller | period = 900\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.322752 | controller | 1 ']'\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.322760 | controller | ']'\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-decision-engine 2026-01-21 11:38:24.322767 | controller | config...'\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | Checking watcher-decision-engine 2026-01-21 11:38:24.322775 | controller | config...\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.322783 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.322791 | controller | | +++ oc rsh -c watcher-decision-engine watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.322799 | controller | \ logger.go:42: 11:24:41 | watcher-tls/2-patch-mtls | error: Internal error occurred: 2026-01-21 11:38:24.322806 | controller | error executing command in container: container is not created or running\n logger.go:42: 2026-01-21 11:38:24.322814 | controller | 11:24:41 | watcher-tls/2-patch-mtls | ++ echo\n logger.go:42: 11:24:41 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.322837 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | running 2026-01-21 11:38:24.322848 | controller | command: [sh -c set -euxo pipefail\n \n oc project ${NAMESPACE}\n 2026-01-21 11:38:24.322856 | controller | \ # Get pod names for each watcher service\n APIPOD=$(oc get pods -l 2026-01-21 11:38:24.322864 | controller | service=watcher-api -o jsonpath='{.items[0].metadata.name}')\n APPLIERPOD=$(oc 2026-01-21 11:38:24.322872 | controller | get pods -l service=watcher-applier -o jsonpath='{.items[0].metadata.name}')\n DECISIONENGINEPOD=$(oc 2026-01-21 11:38:24.322880 | controller | get pods -l service=watcher-decision-engine -o jsonpath='{.items[0].metadata.name}')\n 2026-01-21 11:38:24.322888 | controller | \ \n # Verify memcached mTLS config parameters in watcher-api config\n 2026-01-21 11:38:24.322896 | controller | \ if [ -n \"${APIPOD}\" ]; then\n echo \"Checking watcher-api config...\"\n 2026-01-21 11:38:24.322909 | controller | \ [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.322917 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.322924 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.322935 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.322943 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.322951 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.322959 | controller | $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.322967 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n # Verify 2026-01-21 11:38:24.322975 | controller | mTLS config parameters in memcached backend config\n [ $(echo $(oc rsh 2026-01-21 11:38:24.322982 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.322990 | controller | \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.322997 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.323005 | controller | \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo $(oc rsh 2026-01-21 11:38:24.323012 | controller | -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) | grep -c 2026-01-21 11:38:24.323025 | controller | \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n exit 2026-01-21 11:38:24.323033 | controller | 1\n fi\n \n # Verify memcached mTLS config parameters in watcher-applier 2026-01-21 11:38:24.323041 | controller | config\n if [ -n \"${APPLIERPOD}\" ]; then\n echo \"Checking watcher-applier 2026-01-21 11:38:24.323048 | controller | config...\"\n [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323055 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.323061 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323067 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.323073 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323079 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.323085 | controller | $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323091 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.323097 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323103 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.323109 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323114 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.323120 | controller | $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323126 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.323132 | controller | \ exit 1\n fi\n \n # Verify memcached mTLS config parameters 2026-01-21 11:38:24.323138 | controller | in watcher-decision-engine config\n if [ -n \"${DECISIONENGINEPOD}\" ]; then\n 2026-01-21 11:38:24.323144 | controller | \ echo \"Checking watcher-decision-engine config...\"\n [ $(echo 2026-01-21 11:38:24.323150 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323168 | controller | | grep -c \"memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt \") == 1 ]\n [ 2026-01-21 11:38:24.323175 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323183 | controller | | grep -c \"memcache_tls_keyfile = /etc/pki/tls/private/mtls.key\") == 1 ]\n [ 2026-01-21 11:38:24.323189 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323194 | controller | | grep -c \"memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n [ 2026-01-21 11:38:24.323200 | controller | $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323206 | controller | | grep -c \"memcache_tls_enabled = true\") == 1 ]\n \n [ $(echo 2026-01-21 11:38:24.323212 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323218 | controller | | grep -c \"tls_certfile=/etc/pki/tls/certs/mtls.crt\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.323224 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323230 | controller | | grep -c \"tls_keyfile=/etc/pki/tls/private/mtls.key\") == 1 ]\n [ $(echo 2026-01-21 11:38:24.323240 | controller | $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.323246 | controller | | grep -c \"tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\") == 1 ]\n else\n 2026-01-21 11:38:24.323252 | controller | \ exit 1\n fi\n ]\n logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.323257 | controller | | + oc project watcher-kuttl-default\n logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.323263 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.323269 | controller | \ logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.323275 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.323285 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.323290 | controller | | ++ oc get pods -l service=watcher-applier -o 'jsonpath={.items[0].metadata.name}'\n 2026-01-21 11:38:24.323296 | controller | \ logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0\n 2026-01-21 11:38:24.323302 | controller | \ logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.323308 | controller | -o 'jsonpath={.items[0].metadata.name}'\n logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.323314 | controller | | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0\n logger.go:42: 11:24:43 2026-01-21 11:38:24.323320 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 2026-01-21 11:38:24.323326 | controller | 11:24:43 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-api config...'\n 2026-01-21 11:38:24.323331 | controller | \ logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | Checking watcher-api config...\n 2026-01-21 11:38:24.323337 | controller | \ logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.323343 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.323349 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.323355 | controller | \ logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.323361 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.323367 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.323373 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.323378 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.323384 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.323390 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.323396 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.323402 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.323408 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.323422 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.323429 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.323438 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.323444 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.323450 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.323456 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.323462 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.323468 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.323474 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.323484 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.323494 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.323510 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.323518 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.323525 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.323532 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.323539 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.323546 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.323553 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.323560 | controller | 11:24:43 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:43 2026-01-21 11:38:24.323567 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_keyfile = /etc/pki/tls/private/mtls.key'\n 2026-01-21 11:38:24.323574 | controller | \ logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.323581 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.323589 | controller | 11:24:43 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.323596 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.323602 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.323609 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.323616 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.323625 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.323632 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.323639 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.323646 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.323653 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.323660 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.323676 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.323684 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.323691 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.323698 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.323705 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.323732 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.323763 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.323772 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.323779 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.323786 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.323794 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.323801 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.323808 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.323816 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.323829 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.323837 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.323844 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.323854 | controller | 11:24:43 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:43 2026-01-21 11:38:24.323861 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt'\n 2026-01-21 11:38:24.323869 | controller | \ logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.323876 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.323882 | controller | 11:24:44 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.323888 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.323894 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.323900 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.323906 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.323912 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.323918 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.323924 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.323929 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.323939 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.323945 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.323951 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.323957 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.323963 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.323969 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.323975 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.323980 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.323986 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.323992 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.323998 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.324004 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.324010 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.324016 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.324021 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.324037 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.324046 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.324054 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.324060 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.324068 | controller | 11:24:44 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:44 2026-01-21 11:38:24.324075 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_enabled = true'\n logger.go:42: 2026-01-21 11:38:24.324082 | controller | 11:24:44 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.324088 | controller | cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.324096 | controller | | ++ echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.324103 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.324110 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.324117 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.324124 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.324131 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.324138 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324154 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.324165 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.324173 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.324180 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324187 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324193 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.324200 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324208 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324214 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.324221 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324229 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.324238 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.324246 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.324253 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.324262 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.324271 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.324279 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.324287 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.324295 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.324303 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.324310 | controller | 11:24:44 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:44 2026-01-21 11:38:24.324317 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt\n 2026-01-21 11:38:24.324323 | controller | \ logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.324333 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.324348 | controller | 11:24:44 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.324354 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.324360 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.324366 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.324372 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.324378 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.324384 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.324394 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324400 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.324406 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.324411 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.324417 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324423 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324429 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.324435 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324441 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324447 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.324453 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324459 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.324465 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.324470 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.324476 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.324482 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.324488 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.324494 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.324500 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.324506 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.324511 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.324517 | controller | 11:24:44 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:44 2026-01-21 11:38:24.324523 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key\n 2026-01-21 11:38:24.324529 | controller | \ logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.324535 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.324541 | controller | 11:24:45 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.324547 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.324553 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.324559 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.324564 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.324574 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.324587 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.324594 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324600 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.324605 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.324611 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.324617 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324623 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324629 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.324635 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324641 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324647 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.324653 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324658 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.324664 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.324670 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.324676 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.324682 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.324688 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.324694 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.324699 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.324705 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.324711 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.324736 | controller | 11:24:45 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:45 2026-01-21 11:38:24.324742 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\n 2026-01-21 11:38:24.324748 | controller | \ logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.324754 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.324784 | controller | 11:24:45 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.324790 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.324796 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.324802 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.324808 | controller | = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.324820 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.324826 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.324832 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324838 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.324844 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.324850 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.324855 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324861 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324878 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.324885 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.324891 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.324897 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.324903 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.324909 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.324915 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.324921 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.324927 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.324932 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.324938 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.324944 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.324950 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.324956 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.324962 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.324968 | controller | 11:24:45 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:45 2026-01-21 11:38:24.324977 | controller | | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-applier-0 ']'\n logger.go:42: 2026-01-21 11:38:24.324983 | controller | 11:24:45 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-applier config...'\n 2026-01-21 11:38:24.324989 | controller | \ logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | Checking watcher-applier 2026-01-21 11:38:24.324995 | controller | config...\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.325001 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.325007 | controller | | +++ oc rsh -c watcher-applier watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.325016 | controller | \ logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.325022 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.325029 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.325037 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.325045 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.325053 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.325060 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325067 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.325075 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.325083 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.325091 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325099 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325105 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.325110 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325116 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325122 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.325128 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325145 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.325152 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.325158 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.325164 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.325170 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.325176 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.325183 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.325190 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.325196 | controller | period = 900\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.325202 | controller | 1 ']'\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_keyfile 2026-01-21 11:38:24.325208 | controller | = /etc/pki/tls/private/mtls.key'\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.325214 | controller | | +++ oc rsh -c watcher-applier watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.325220 | controller | \ logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.325232 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.325238 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.325244 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.325250 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.325256 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.325262 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325268 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.325274 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.325283 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.325289 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325298 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325304 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.325310 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325316 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325321 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.325327 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325333 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.325339 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.325345 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.325351 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.325357 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.325362 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.325368 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.325374 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.325380 | controller | period = 900\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.325394 | controller | 1 ']'\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_cafile 2026-01-21 11:38:24.325401 | controller | = /etc/pki/tls/certs/mtls-ca.crt'\n logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.325407 | controller | | +++ oc rsh -c watcher-applier watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.325413 | controller | \ logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.325421 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.325431 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.325437 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.325442 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.325448 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.325454 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325460 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.325466 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.325472 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.325477 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325483 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325489 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.325495 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325501 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325507 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.325512 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325518 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.325524 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.325530 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.325536 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.325541 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.325547 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.325553 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.325559 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.325565 | controller | period = 900\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.325571 | controller | 1 ']'\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_enabled 2026-01-21 11:38:24.325576 | controller | = true'\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.325582 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.325588 | controller | 11:24:46 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.325594 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.325600 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.325609 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.325614 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.325630 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.325636 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325642 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.325648 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.325654 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.325660 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325665 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325671 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.325677 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325683 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325689 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.325695 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325700 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.325706 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.325732 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.325739 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.325744 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.325750 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.325756 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.325762 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.325768 | controller | period = 900\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.325774 | controller | 1 ']'\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt\n 2026-01-21 11:38:24.325780 | controller | \ logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.325785 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.325791 | controller | 11:24:46 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.325797 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.325803 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.325809 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.325818 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.325827 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.325833 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325839 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.325845 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.325851 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.325857 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325863 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325869 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.325875 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.325891 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.325897 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.325906 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.325912 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.325918 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.325924 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.325930 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.325936 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.325942 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.325947 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.325953 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.325959 | controller | period = 900\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.325965 | controller | 1 ']'\n logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key\n 2026-01-21 11:38:24.325971 | controller | \ logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.325980 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.325986 | controller | 11:24:47 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.325992 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.325997 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.326003 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.326013 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.326018 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.326024 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326030 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.326036 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.326042 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.326048 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326054 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326059 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.326065 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326071 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326077 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.326083 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326089 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.326094 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.326100 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.326106 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.326112 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.326118 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.326124 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.326142 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.326149 | controller | period = 900\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.326155 | controller | 1 ']'\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\n 2026-01-21 11:38:24.326161 | controller | \ logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.326167 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 2026-01-21 11:38:24.326172 | controller | 11:24:47 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher 2026-01-21 11:38:24.326178 | controller | transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.326184 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.326194 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.326200 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.326209 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.326215 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326221 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.326228 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.326235 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.326241 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326247 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326253 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.326261 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326267 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326274 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.326281 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326288 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.326293 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.326299 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[cinder_client]' 2026-01-21 11:38:24.326308 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.326314 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.326320 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.326326 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.326332 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.326338 | controller | period = 900\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + '[' 1 == 2026-01-21 11:38:24.326344 | controller | 1 ']'\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + '[' -n watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.326350 | controller | ']'\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + echo 'Checking watcher-decision-engine 2026-01-21 11:38:24.326355 | controller | config...'\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | Checking watcher-decision-engine 2026-01-21 11:38:24.326361 | controller | config...\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | +++ oc rsh -c 2026-01-21 11:38:24.326367 | controller | watcher-decision-engine watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.326373 | controller | \ logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_certfile 2026-01-21 11:38:24.326379 | controller | = /etc/pki/tls/certs/mtls.crt '\n logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.326392 | controller | | ++ echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.326398 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.326410 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.326416 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.326422 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.326428 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326434 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.326440 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.326446 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.326452 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326458 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326463 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.326469 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326475 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326484 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.326490 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326495 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.326501 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.326507 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.326513 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.326519 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.326525 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.326531 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.326537 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.326543 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.326548 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.326572 | controller | 11:24:47 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:47 2026-01-21 11:38:24.326580 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_keyfile = /etc/pki/tls/private/mtls.key'\n 2026-01-21 11:38:24.326588 | controller | \ logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.326601 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.326616 | controller | \ logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.326624 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.326639 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.326649 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.326665 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.326673 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.326680 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326704 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.326730 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.326742 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.326750 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326761 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326777 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.326786 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326794 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326801 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.326809 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326817 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.326824 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.326831 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.326838 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.326845 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.326852 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.326859 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.326865 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.326871 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.326877 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.326883 | controller | 11:24:48 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:48 2026-01-21 11:38:24.326889 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt'\n 2026-01-21 11:38:24.326895 | controller | \ logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.326906 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.326912 | controller | \ logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.326921 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.326927 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.326933 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.326939 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.326945 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.326951 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.326957 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.326963 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.326969 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.326975 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.326981 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.326987 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.326992 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327011 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327020 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.327028 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327036 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.327045 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.327052 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.327059 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.327066 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.327073 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.327084 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.327100 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.327108 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.327116 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.327124 | controller | 11:24:48 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:48 2026-01-21 11:38:24.327131 | controller | | watcher-tls/2-patch-mtls | ++ grep -c 'memcache_tls_enabled = true'\n logger.go:42: 2026-01-21 11:38:24.327138 | controller | 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.327145 | controller | cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.327158 | controller | | ++ echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.327166 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.327174 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.327184 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.327193 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.327202 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327215 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.327224 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.327232 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.327240 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327247 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327255 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.327262 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327270 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327277 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.327284 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327291 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.327297 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.327303 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.327309 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.327315 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.327332 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.327348 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.327361 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.327369 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.327377 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.327385 | controller | 11:24:48 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:48 2026-01-21 11:38:24.327392 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt\n 2026-01-21 11:38:24.327399 | controller | \ logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.327406 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.327419 | controller | \ logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.327425 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.327431 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.327437 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.327443 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.327449 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.327455 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327461 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.327466 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.327472 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.327478 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327484 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327490 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.327496 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327502 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327508 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.327513 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327524 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.327530 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.327536 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.327542 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.327547 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.327553 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.327559 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.327565 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.327571 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.327577 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.327583 | controller | 11:24:48 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:48 2026-01-21 11:38:24.327589 | controller | | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key\n 2026-01-21 11:38:24.327595 | controller | \ logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.327618 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.327636 | controller | \ logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path 2026-01-21 11:38:24.327646 | controller | = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.327654 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.327661 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.327669 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.327676 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.327684 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327696 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.327728 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.327744 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.327753 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327760 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327768 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.327775 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.327782 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.327789 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.327796 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.327804 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.327817 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.327837 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.327845 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.327853 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.327860 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.327868 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.327875 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.327882 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.327889 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.327897 | controller | 11:24:49 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:49 2026-01-21 11:38:24.327904 | controller | | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.327920 | controller | cat /etc/watcher/watcher.conf.d/00-default.conf\n logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls 2026-01-21 11:38:24.327928 | controller | | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt\n logger.go:42: 11:24:49 2026-01-21 11:38:24.327935 | controller | | watcher-tls/2-patch-mtls | ++ echo '[DEFAULT]' state_path = /var/lib/watcher transport_url 2026-01-21 11:38:24.327943 | controller | = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.327950 | controller | control_exchange = watcher debug = True '#' empty notification_level means that 2026-01-21 11:38:24.327957 | controller | no notification will be sent notification_level = '[database]' connection = 'mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.327964 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.327988 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.327998 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.328008 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.328015 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.328022 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.328029 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.328037 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.328045 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.328053 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.328063 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.328080 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.328096 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.328104 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.328111 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.328119 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.328126 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.328133 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.328140 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.328151 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.328172 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.328180 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.328187 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.328195 | controller | 11:24:49 | watcher-tls/2-patch-mtls | + '[' 1 == 1 ']'\n logger.go:42: 11:24:49 2026-01-21 11:38:24.328211 | controller | | watcher-tls/2-patch-mtls | test step completed 2-patch-mtls\n logger.go:42: 2026-01-21 11:38:24.328219 | controller | 11:24:49 | watcher-tls/3-disable-podlevel-tls | starting test step 3-disable-podlevel-tls\n 2026-01-21 11:38:24.328226 | controller | \ logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.328236 | controller | [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type='json' -p='[{\"op\": 2026-01-21 11:38:24.328244 | controller | \"replace\", \"path\": \"/spec/apiServiceTemplate/tls/api\", \"value\":{ \"internal\": 2026-01-21 11:38:24.328251 | controller | {}, \"public\": {} }}]'\n ]\n logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328260 | controller | | watcher.watcher.openstack.org/watcher-kuttl patched\n logger.go:42: 11:24:49 2026-01-21 11:38:24.328269 | controller | | watcher-tls/3-disable-podlevel-tls | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.328278 | controller | \ oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.328288 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.328295 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID | grep 2026-01-21 11:38:24.328302 | controller | watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} keystoneservice watcher 2026-01-21 11:38:24.328308 | controller | -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.328314 | controller | watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" ]\n # check that 2026-01-21 11:38:24.328320 | controller | watcher internal endpoint does not use https\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.328326 | controller | -- openstack endpoint list | grep infra-optim | grep internal | [ $(grep -c https) 2026-01-21 11:38:24.328332 | controller | == 0 ]\n # check that watcher public endpoint does use https\n oc 2026-01-21 11:38:24.328338 | controller | exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep infra-optim 2026-01-21 11:38:24.328344 | controller | | grep public | [ $(grep -c https) == 1 ]\n # If we are running the container 2026-01-21 11:38:24.328360 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.328378 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.328389 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.328397 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.328405 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.328412 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.328418 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.328424 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.328430 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.328436 | controller | \ ]\n logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | + oc 2026-01-21 11:38:24.328442 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.328447 | controller | -c Name -c Type\n logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328453 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328459 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328465 | controller | | ++ grep watcher\n logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328484 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328490 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.328500 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328507 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328513 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.328518 | controller | \ logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.328524 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328530 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.328536 | controller | \ logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.328542 | controller | ']'\n logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + grep internal\n 2026-01-21 11:38:24.328548 | controller | \ logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim\n 2026-01-21 11:38:24.328554 | controller | \ logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.328560 | controller | openstackclient -- openstack endpoint list\n logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328566 | controller | | ++ grep -c https\n logger.go:42: 11:24:59 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328572 | controller | | + '[' 1 == 0 ']'\n logger.go:42: 11:25:00 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328578 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.328583 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.328589 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.328595 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.328601 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.328607 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.328613 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # check that watcher internal endpoint 2026-01-21 11:38:24.328619 | controller | does not use https\n oc exec -n ${NAMESPACE} openstackclient -- openstack 2026-01-21 11:38:24.328625 | controller | endpoint list | grep infra-optim | grep internal | [ $(grep -c https) == 0 ]\n 2026-01-21 11:38:24.328630 | controller | \ # check that watcher public endpoint does use https\n oc exec -n 2026-01-21 11:38:24.328636 | controller | ${NAMESPACE} openstackclient -- openstack endpoint list | grep infra-optim | grep 2026-01-21 11:38:24.328642 | controller | public | [ $(grep -c https) == 1 ]\n # If we are running the container locally, 2026-01-21 11:38:24.328660 | controller | skip following test\n if [ \"$(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.328669 | controller | -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 0\n 2026-01-21 11:38:24.328677 | controller | \ fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.328685 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.328693 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.328705 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.328730 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.328743 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.328751 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.328760 | controller | \ ]\n logger.go:42: 11:25:00 | watcher-tls/3-disable-podlevel-tls | + oc 2026-01-21 11:38:24.328766 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.328772 | controller | -c Name -c Type\n logger.go:42: 11:25:00 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328778 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328784 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328790 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.328795 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328801 | controller | | ++ grep watcher\n logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328807 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328813 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328823 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.328829 | controller | \ logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.328835 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328841 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.328847 | controller | \ logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.328853 | controller | ']'\n logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim\n 2026-01-21 11:38:24.328862 | controller | \ logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + grep internal\n 2026-01-21 11:38:24.328868 | controller | \ logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.328874 | controller | openstackclient -- openstack endpoint list\n logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328880 | controller | | ++ grep -c https\n logger.go:42: 11:25:08 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328885 | controller | | + '[' 1 == 0 ']'\n logger.go:42: 11:25:09 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.328891 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.328897 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.328903 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.328909 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.328917 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.328932 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.328944 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # check that watcher internal endpoint 2026-01-21 11:38:24.328957 | controller | does not use https\n oc exec -n ${NAMESPACE} openstackclient -- openstack 2026-01-21 11:38:24.328965 | controller | endpoint list | grep infra-optim | grep internal | [ $(grep -c https) == 0 ]\n 2026-01-21 11:38:24.328973 | controller | \ # check that watcher public endpoint does use https\n oc exec -n 2026-01-21 11:38:24.328980 | controller | ${NAMESPACE} openstackclient -- openstack endpoint list | grep infra-optim | grep 2026-01-21 11:38:24.328988 | controller | public | [ $(grep -c https) == 1 ]\n # If we are running the container locally, 2026-01-21 11:38:24.328995 | controller | skip following test\n if [ \"$(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.329015 | controller | -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 0\n 2026-01-21 11:38:24.329024 | controller | \ fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.329039 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.329052 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.329060 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.329074 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.329087 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.329095 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.329103 | controller | \ ]\n logger.go:42: 11:25:09 | watcher-tls/3-disable-podlevel-tls | + oc 2026-01-21 11:38:24.329110 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.329117 | controller | -c Name -c Type\n logger.go:42: 11:25:09 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329125 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329132 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329139 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.329147 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329154 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329163 | controller | | ++ grep watcher\n logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329170 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329180 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.329188 | controller | \ logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.329196 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329203 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.329216 | controller | \ logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.329223 | controller | ']'\n logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + grep internal\n 2026-01-21 11:38:24.329230 | controller | \ logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim\n 2026-01-21 11:38:24.329238 | controller | \ logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.329250 | controller | openstackclient -- openstack endpoint list\n logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329262 | controller | | ++ grep -c https\n logger.go:42: 11:25:17 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329270 | controller | | + '[' 1 == 0 ']'\n logger.go:42: 11:25:18 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329277 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.329285 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.329292 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.329299 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.329306 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.329313 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.329320 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # check that watcher internal endpoint 2026-01-21 11:38:24.329328 | controller | does not use https\n oc exec -n ${NAMESPACE} openstackclient -- openstack 2026-01-21 11:38:24.329335 | controller | endpoint list | grep infra-optim | grep internal | [ $(grep -c https) == 0 ]\n 2026-01-21 11:38:24.329342 | controller | \ # check that watcher public endpoint does use https\n oc exec -n 2026-01-21 11:38:24.329350 | controller | ${NAMESPACE} openstackclient -- openstack endpoint list | grep infra-optim | grep 2026-01-21 11:38:24.329358 | controller | public | [ $(grep -c https) == 1 ]\n # If we are running the container locally, 2026-01-21 11:38:24.329365 | controller | skip following test\n if [ \"$(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.329388 | controller | -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 0\n 2026-01-21 11:38:24.329398 | controller | \ fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.329406 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.329414 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.329422 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.329430 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.329437 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.329444 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.329451 | controller | \ ]\n logger.go:42: 11:25:18 | watcher-tls/3-disable-podlevel-tls | + oc 2026-01-21 11:38:24.329458 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.329466 | controller | -c Name -c Type\n logger.go:42: 11:25:18 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329474 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329481 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329488 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.329495 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329503 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329510 | controller | | ++ grep watcher\n logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329517 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329531 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.329539 | controller | \ logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.329547 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329554 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.329562 | controller | \ logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.329570 | controller | ']'\n logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim\n 2026-01-21 11:38:24.329577 | controller | \ logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + grep internal\n 2026-01-21 11:38:24.329584 | controller | \ logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https\n 2026-01-21 11:38:24.329592 | controller | \ logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.329599 | controller | openstackclient -- openstack endpoint list\n logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329607 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329622 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack endpoint list\n 2026-01-21 11:38:24.329630 | controller | \ logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | + grep public\n 2026-01-21 11:38:24.329638 | controller | \ logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim\n 2026-01-21 11:38:24.329645 | controller | \ logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https\n 2026-01-21 11:38:24.329653 | controller | \ logger.go:42: 11:25:31 | watcher-tls/3-disable-podlevel-tls | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.329661 | controller | \ logger.go:42: 11:25:31 | watcher-tls/3-disable-podlevel-tls | ++ oc get pods 2026-01-21 11:38:24.329669 | controller | -n openstack-operators -o name -l openstack.org/operator-name=watcher\n logger.go:42: 2026-01-21 11:38:24.329676 | controller | 11:25:32 | watcher-tls/3-disable-podlevel-tls | + '[' '' == '' ']'\n logger.go:42: 2026-01-21 11:38:24.329683 | controller | 11:25:32 | watcher-tls/3-disable-podlevel-tls | + exit 0\n logger.go:42: 11:25:33 2026-01-21 11:38:24.329691 | controller | | watcher-tls/3-disable-podlevel-tls | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.329699 | controller | \ oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.329706 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.329733 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID | grep 2026-01-21 11:38:24.329750 | controller | watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} keystoneservice watcher 2026-01-21 11:38:24.329767 | controller | -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.329775 | controller | watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" ]\n # check that 2026-01-21 11:38:24.329783 | controller | watcher internal endpoint does not use https\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.329791 | controller | -- openstack endpoint list | grep infra-optim | grep internal | [ $(grep -c https) 2026-01-21 11:38:24.329800 | controller | == 0 ]\n # check that watcher public endpoint does use https\n oc 2026-01-21 11:38:24.329808 | controller | exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep infra-optim 2026-01-21 11:38:24.329815 | controller | | grep public | [ $(grep -c https) == 1 ]\n # If we are running the container 2026-01-21 11:38:24.329823 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.329836 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.329844 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.329852 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.329860 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.329873 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.329881 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.329890 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.329898 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.329905 | controller | \ ]\n logger.go:42: 11:25:33 | watcher-tls/3-disable-podlevel-tls | + oc 2026-01-21 11:38:24.329913 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.329921 | controller | -c Name -c Type\n logger.go:42: 11:25:33 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329929 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329938 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329947 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.329955 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329968 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329976 | controller | | ++ grep watcher\n logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329984 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.329992 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.330000 | controller | \ logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.330007 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.330014 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.330022 | controller | \ logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.330030 | controller | ']'\n logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + oc exec 2026-01-21 11:38:24.330038 | controller | -n watcher-kuttl-default openstackclient -- openstack endpoint list\n logger.go:42: 2026-01-21 11:38:24.330046 | controller | 11:25:39 | watcher-tls/3-disable-podlevel-tls | + grep internal\n logger.go:42: 2026-01-21 11:38:24.330053 | controller | 11:25:39 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim\n logger.go:42: 2026-01-21 11:38:24.330061 | controller | 11:25:39 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https\n logger.go:42: 2026-01-21 11:38:24.330068 | controller | 11:25:42 | watcher-tls/3-disable-podlevel-tls | + '[' 0 == 0 ']'\n logger.go:42: 2026-01-21 11:38:24.330076 | controller | 11:25:42 | watcher-tls/3-disable-podlevel-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.330083 | controller | openstackclient -- openstack endpoint list\n logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.330091 | controller | | + grep infra-optim\n logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.330109 | controller | | + grep public\n logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.330116 | controller | | ++ grep -c https\n logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.330137 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls 2026-01-21 11:38:24.330153 | controller | | ++ oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n 2026-01-21 11:38:24.330166 | controller | \ logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | + '[' '' == '' 2026-01-21 11:38:24.330175 | controller | ']'\n logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | + exit 0\n 2026-01-21 11:38:24.330182 | controller | \ logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | test step completed 2026-01-21 11:38:24.330188 | controller | 3-disable-podlevel-tls\n logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330194 | controller | | starting test step 4-deploy-without-route\n logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330200 | controller | | running command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type='json' 2026-01-21 11:38:24.330206 | controller | -p='[{\"op\": \"replace\", \"path\": \"/spec/apiServiceTemplate/override\", \"value\":{\"service\": 2026-01-21 11:38:24.330212 | controller | { \"internal\": {}, \"public\": { \"metadata\": { \"annotations\": { \"metallb.universe.tf/address-pool\": 2026-01-21 11:38:24.330218 | controller | \"ctlplane\", \"metallb.universe.tf/allow-shared-ip\": \"ctlplane\" } }, \"spec\": 2026-01-21 11:38:24.330224 | controller | { \"type\": \"LoadBalancer\" } } } }}]'\n ]\n logger.go:42: 11:25:45 | 2026-01-21 11:38:24.330231 | controller | watcher-tls/4-deploy-without-route | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.330238 | controller | patched\n logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | running 2026-01-21 11:38:24.330244 | controller | command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.330250 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.330256 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.330264 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.330270 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.330277 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.330284 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.330290 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.330301 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.330307 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.330313 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.330319 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.330325 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.330331 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.330337 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.330343 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.330349 | controller | \ ]\n logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | + oc 2026-01-21 11:38:24.330355 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.330364 | controller | -c Name -c Type\n logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330370 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330376 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330382 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.330388 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330394 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330400 | controller | | ++ grep watcher\n logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330406 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330412 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.330417 | controller | \ logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.330423 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330429 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.330447 | controller | \ logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.330456 | controller | ']'\n logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | ++ oc get 2026-01-21 11:38:24.330464 | controller | pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n logger.go:42: 2026-01-21 11:38:24.330472 | controller | 11:25:51 | watcher-tls/4-deploy-without-route | + '[' '' == '' ']'\n logger.go:42: 2026-01-21 11:38:24.330479 | controller | 11:25:51 | watcher-tls/4-deploy-without-route | + exit 0\n logger.go:42: 11:25:52 2026-01-21 11:38:24.330490 | controller | | watcher-tls/4-deploy-without-route | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.330506 | controller | \ oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.330514 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.330522 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID | grep 2026-01-21 11:38:24.330535 | controller | watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} keystoneservice watcher 2026-01-21 11:38:24.330548 | controller | -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.330574 | controller | watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" ]\n # If we are 2026-01-21 11:38:24.330581 | controller | running the container locally, skip following test\n if [ \"$(oc get pods 2026-01-21 11:38:24.330588 | controller | -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" == \"\" 2026-01-21 11:38:24.330595 | controller | ]; then\n exit 0\n fi\n env_variables=$(oc set env $(oc 2026-01-21 11:38:24.330602 | controller | get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.330609 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.330616 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.330624 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.330631 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.330645 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.330652 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:25:52 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330660 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.330667 | controller | value -c Name -c Type\n logger.go:42: 11:25:52 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330687 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330700 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330708 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.330750 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330764 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330771 | controller | | ++ grep watcher\n logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330778 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330785 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.330792 | controller | \ logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.330800 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route 2026-01-21 11:38:24.330807 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.330814 | controller | \ logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.330822 | controller | ']'\n logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | ++ oc get 2026-01-21 11:38:24.330829 | controller | pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n logger.go:42: 2026-01-21 11:38:24.330837 | controller | 11:25:57 | watcher-tls/4-deploy-without-route | + '[' '' == '' ']'\n logger.go:42: 2026-01-21 11:38:24.330844 | controller | 11:25:57 | watcher-tls/4-deploy-without-route | + exit 0\n logger.go:42: 11:25:57 2026-01-21 11:38:24.330851 | controller | | watcher-tls/4-deploy-without-route | test step completed 4-deploy-without-route\n 2026-01-21 11:38:24.330869 | controller | \ logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | starting test step 5-disable-tls\n 2026-01-21 11:38:24.330878 | controller | \ logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | running command: [sh -c 2026-01-21 11:38:24.330885 | controller | oc patch watcher -n $NAMESPACE watcher-kuttl --type='json' -p='[{\"op\": \"replace\", 2026-01-21 11:38:24.330892 | controller | \"path\": \"/spec/apiServiceTemplate/override\", \"value\":{}}]'\n ]\n logger.go:42: 2026-01-21 11:38:24.330899 | controller | 11:25:57 | watcher-tls/5-disable-tls | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.330907 | controller | patched\n logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | running command: 2026-01-21 11:38:24.330914 | controller | [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient -- openstack 2026-01-21 11:38:24.330928 | controller | service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc 2026-01-21 11:38:24.330936 | controller | exec -n ${NAMESPACE} openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.330944 | controller | -c Type -c ID | grep watcher| awk '{print $1}')\n [ $(oc get -n ${NAMESPACE} 2026-01-21 11:38:24.330955 | controller | keystoneservice watcher -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ 2026-01-21 11:38:24.330977 | controller | -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" 2026-01-21 11:38:24.330985 | controller | ]\n # If we are running the container locally, skip following test\n if 2026-01-21 11:38:24.330992 | controller | [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.330999 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.331007 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.331014 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.331021 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.331029 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.331037 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.331044 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.331051 | controller | variables found.\"\n fi\n # check that no watcher endpoint uses https\n 2026-01-21 11:38:24.331060 | controller | \ oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.331067 | controller | infra-optim | [ $(grep -c https) == 0 ]\n ]\n logger.go:42: 11:25:57 | 2026-01-21 11:38:24.331075 | controller | watcher-tls/5-disable-tls | + oc exec -n watcher-kuttl-default openstackclient -- 2026-01-21 11:38:24.331082 | controller | openstack service list -f value -c Name -c Type\n logger.go:42: 11:25:57 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331089 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:25:59 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331096 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | ++ 2026-01-21 11:38:24.331104 | controller | oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.331117 | controller | -c Name -c Type -c ID\n logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | 2026-01-21 11:38:24.331125 | controller | ++ awk '{print $1}'\n logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | ++ 2026-01-21 11:38:24.331133 | controller | grep watcher\n logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n 2026-01-21 11:38:24.331140 | controller | \ logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.331147 | controller | keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n logger.go:42: 11:26:02 2026-01-21 11:38:24.331155 | controller | | watcher-tls/5-disable-tls | + '[' eee10af9590f44de87d443cfc0f41a86 == eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.331164 | controller | ']'\n logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.331172 | controller | watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n logger.go:42: 11:26:02 2026-01-21 11:38:24.331180 | controller | | watcher-tls/5-disable-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.331189 | controller | ']'\n logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | ++ oc get pods -n 2026-01-21 11:38:24.331195 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher\n logger.go:42: 2026-01-21 11:38:24.331201 | controller | 11:26:02 | watcher-tls/5-disable-tls | + '[' '' == '' ']'\n logger.go:42: 11:26:02 2026-01-21 11:38:24.331207 | controller | | watcher-tls/5-disable-tls | + exit 0\n logger.go:42: 11:26:03 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331213 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n ${NAMESPACE} openstackclient 2026-01-21 11:38:24.331231 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.331255 | controller | \ SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.331266 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ 2026-01-21 11:38:24.331274 | controller | $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.331281 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n ${NAMESPACE} watcher watcher-kuttl 2026-01-21 11:38:24.331287 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.331293 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.331306 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.331320 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.331330 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.331338 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.331346 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.331354 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.331361 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.331368 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.331374 | controller | \ # check that no watcher endpoint uses https\n oc exec -n ${NAMESPACE} 2026-01-21 11:38:24.331380 | controller | openstackclient -- openstack endpoint list | grep infra-optim | [ $(grep -c https) 2026-01-21 11:38:24.331386 | controller | == 0 ]\n ]\n logger.go:42: 11:26:03 | watcher-tls/5-disable-tls | ++ grep 2026-01-21 11:38:24.331392 | controller | -c '^watcher'\n logger.go:42: 11:26:03 | watcher-tls/5-disable-tls | + oc exec 2026-01-21 11:38:24.331398 | controller | -n watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.331404 | controller | -c Type\n logger.go:42: 11:26:06 | watcher-tls/5-disable-tls | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.331409 | controller | \ logger.go:42: 11:26:06 | watcher-tls/5-disable-tls | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.331415 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID\n logger.go:42: 2026-01-21 11:38:24.331421 | controller | 11:26:06 | watcher-tls/5-disable-tls | ++ awk '{print $1}'\n logger.go:42: 11:26:06 2026-01-21 11:38:24.331427 | controller | | watcher-tls/5-disable-tls | ++ grep watcher\n logger.go:42: 11:26:08 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331433 | controller | | + SERVICEID=eee10af9590f44de87d443cfc0f41a86\n logger.go:42: 11:26:08 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331439 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.331444 | controller | \ logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | + '[' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.331450 | controller | == eee10af9590f44de87d443cfc0f41a86 ']'\n logger.go:42: 11:26:08 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331456 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.331462 | controller | \ logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.331468 | controller | ']'\n logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | ++ oc get pods -n 2026-01-21 11:38:24.331474 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher\n logger.go:42: 2026-01-21 11:38:24.331485 | controller | 11:26:09 | watcher-tls/5-disable-tls | + '[' '' == '' ']'\n logger.go:42: 11:26:09 2026-01-21 11:38:24.331491 | controller | | watcher-tls/5-disable-tls | + exit 0\n logger.go:42: 11:26:09 | watcher-tls/5-disable-tls 2026-01-21 11:38:24.331498 | controller | | test step completed 5-disable-tls\n logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher 2026-01-21 11:38:24.331505 | controller | | starting test step 6-cleanup-watcher\n logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher 2026-01-21 11:38:24.331514 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.331521 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.331534 | controller | ]\n ]\n logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher | + oc 2026-01-21 11:38:24.331542 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.331549 | controller | -c Name -c Type\n logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher | ++ 2026-01-21 11:38:24.331557 | controller | grep -c '^watcher'\n logger.go:42: 11:26:12 | watcher-tls/6-cleanup-watcher | 2026-01-21 11:38:24.331621 | controller | + '[' 1 == 0 ']'\n logger.go:42: 11:26:13 | watcher-tls/6-cleanup-watcher | running 2026-01-21 11:38:24.331641 | controller | command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.331648 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.331654 | controller | ]\n ]\n logger.go:42: 11:26:13 | watcher-tls/6-cleanup-watcher | + oc 2026-01-21 11:38:24.331659 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.331665 | controller | -c Name -c Type\n logger.go:42: 11:26:13 | watcher-tls/6-cleanup-watcher | ++ 2026-01-21 11:38:24.331671 | controller | grep -c '^watcher'\n logger.go:42: 11:26:17 | watcher-tls/6-cleanup-watcher | 2026-01-21 11:38:24.331677 | controller | + '[' 0 == 0 ']'\n logger.go:42: 11:26:17 | watcher-tls/6-cleanup-watcher | test 2026-01-21 11:38:24.331683 | controller | step completed 6-cleanup-watcher\n logger.go:42: 11:26:17 | watcher-tls/7-cleanup-certs 2026-01-21 11:38:24.331689 | controller | | starting test step 7-cleanup-certs\n logger.go:42: 11:26:17 | watcher-tls/7-cleanup-certs 2026-01-21 11:38:24.331695 | controller | | test step completed 7-cleanup-certs\n logger.go:42: 11:26:17 | watcher-tls 2026-01-21 11:38:24.331701 | controller | | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-rmquser\n logger.go:42: 2026-01-21 11:38:24.331706 | controller | 11:26:17 | watcher-rmquser | Skipping creation of user-supplied namespace: watcher-kuttl-default\n 2026-01-21 11:38:24.331727 | controller | \ logger.go:42: 11:26:17 | watcher-rmquser/0-cleanup-watcher | starting test step 2026-01-21 11:38:24.331738 | controller | 0-cleanup-watcher\n logger.go:42: 11:26:17 | watcher-rmquser/0-cleanup-watcher 2026-01-21 11:38:24.331749 | controller | | test step completed 0-cleanup-watcher\n logger.go:42: 11:26:17 | watcher-rmquser/1-deploy 2026-01-21 11:38:24.331755 | controller | | starting test step 1-deploy\n logger.go:42: 11:26:17 | watcher-rmquser/1-deploy 2026-01-21 11:38:24.331761 | controller | | Watcher:watcher-kuttl-default/watcher-kuttl created\n logger.go:42: 11:26:17 2026-01-21 11:38:24.331767 | controller | | watcher-rmquser/1-deploy | running command: [sh -c set -euxo pipefail\n \n 2026-01-21 11:38:24.331773 | controller | \ # Wait for Watcher to be Ready\n kubectl wait --for=condition=Ready 2026-01-21 11:38:24.331779 | controller | watcher/watcher-kuttl -n $NAMESPACE --timeout=300s\n \n # Verify WatcherNotificationTransportURLReady 2026-01-21 11:38:24.331784 | controller | condition exists and is True\n kubectl get watcher watcher-kuttl -n $NAMESPACE 2026-01-21 11:38:24.331790 | controller | -o jsonpath='{.status.conditions[?(@.type==\"WatcherNotificationTransportURLReady\")].status}' 2026-01-21 11:38:24.331796 | controller | | grep -q \"True\"\n echo \"WatcherNotificationTransportURLReady condition 2026-01-21 11:38:24.331807 | controller | is True\"\n \n # Count TransportURL CRs - should be exactly 2 (one 2026-01-21 11:38:24.331812 | controller | for messaging, one for notifications)\n transport_count=$(kubectl get transporturl 2026-01-21 11:38:24.331818 | controller | -n $NAMESPACE -o name | grep \"watcher-kuttl-watcher-transport\" | wc -l)\n notification_transport_count=$(kubectl 2026-01-21 11:38:24.331824 | controller | get transporturl -n $NAMESPACE -o name | grep \"watcher-kuttl-watcher-notification\" 2026-01-21 11:38:24.331830 | controller | | wc -l)\n \n if [ \"$transport_count\" -ne \"1\" ]; then\n echo 2026-01-21 11:38:24.331836 | controller | \"Expected 1 watcher-transport TransportURL, found $transport_count\"\n exit 2026-01-21 11:38:24.331842 | controller | 1\n fi\n \n if [ \"$notification_transport_count\" -ne \"1\" 2026-01-21 11:38:24.331848 | controller | ]; then\n echo \"Expected 1 notification-transport TransportURL, found 2026-01-21 11:38:24.331853 | controller | $notification_transport_count\"\n exit 1\n fi\n \n echo 2026-01-21 11:38:24.331859 | controller | \"Correctly found 2 TransportURLs (separate clusters: transport and notification)\"\n 2026-01-21 11:38:24.331865 | controller | \ \n # Verify watcher-transport has correct user and vhost\n transport_user=$(kubectl 2026-01-21 11:38:24.331871 | controller | get transporturl watcher-kuttl-watcher-transport -n $NAMESPACE -o jsonpath='{.spec.username}')\n 2026-01-21 11:38:24.331877 | controller | \ transport_vhost=$(kubectl get transporturl watcher-kuttl-watcher-transport 2026-01-21 11:38:24.331883 | controller | -n $NAMESPACE -o jsonpath='{.spec.vhost}')\n if [ \"$transport_user\" != 2026-01-21 11:38:24.331889 | controller | \"watcher-rpc\" ]; then\n echo \"Expected watcher-transport username 'watcher-rpc', 2026-01-21 11:38:24.331895 | controller | found '$transport_user'\"\n exit 1\n fi\n if [ \"$transport_vhost\" 2026-01-21 11:38:24.331900 | controller | != \"watcher-rpc\" ]; then\n echo \"Expected watcher-transport vhost 'watcher-rpc', 2026-01-21 11:38:24.331909 | controller | found '$transport_vhost'\"\n exit 1\n fi\n echo \"Watcher 2026-01-21 11:38:24.331915 | controller | transport has correct user (watcher-rpc) and vhost (watcher-rpc)\"\n \n # 2026-01-21 11:38:24.331931 | controller | Verify notification-transport has correct user and vhost\n notif_user=$(kubectl 2026-01-21 11:38:24.331940 | controller | get transporturl watcher-kuttl-watcher-notification-rabbitmq-notifications -n $NAMESPACE 2026-01-21 11:38:24.331948 | controller | -o jsonpath='{.spec.username}')\n notif_vhost=$(kubectl get transporturl 2026-01-21 11:38:24.331955 | controller | watcher-kuttl-watcher-notification-rabbitmq-notifications -n $NAMESPACE -o jsonpath='{.spec.vhost}')\n 2026-01-21 11:38:24.331963 | controller | \ if [ \"$notif_user\" != \"watcher-notifications\" ]; then\n echo 2026-01-21 11:38:24.331971 | controller | \"Expected notification-transport username 'watcher-notifications', found '$notif_user'\"\n 2026-01-21 11:38:24.331985 | controller | \ exit 1\n fi\n if [ \"$notif_vhost\" != \"watcher-notifications\" 2026-01-21 11:38:24.331991 | controller | ]; then\n echo \"Expected notification-transport vhost 'watcher-notifications', 2026-01-21 11:38:24.331997 | controller | found '$notif_vhost'\"\n exit 1\n fi\n echo \"Notification 2026-01-21 11:38:24.332003 | controller | transport has correct user (watcher-notifications) and vhost (watcher-notifications)\"\n 2026-01-21 11:38:24.332009 | controller | \ \n # Verify that watcher.conf contains the notifications transport_url\n 2026-01-21 11:38:24.332015 | controller | \ WATCHER_API_POD=$(kubectl get pods -n $NAMESPACE -l \"service=watcher-api\" 2026-01-21 11:38:24.332021 | controller | -o custom-columns=:metadata.name --no-headers | grep -v ^$ | head -1)\n if 2026-01-21 11:38:24.332026 | controller | [ -z \"${WATCHER_API_POD}\" ]; then\n echo \"No watcher-api pod found\"\n 2026-01-21 11:38:24.332039 | controller | \ exit 1\n fi\n # Verify RPC transport_url in DEFAULT section\n 2026-01-21 11:38:24.332045 | controller | \ rpc_transport_url=$(kubectl exec -n $NAMESPACE ${WATCHER_API_POD} -c watcher-api 2026-01-21 11:38:24.332051 | controller | -- cat /etc/watcher/watcher.conf.d/00-default.conf | grep -E '^\\[DEFAULT\\]' -A 2026-01-21 11:38:24.332057 | controller | 50 | grep 'transport_url' | head -1 || true)\n if [ -z \"$rpc_transport_url\" 2026-01-21 11:38:24.332063 | controller | ]; then\n echo \"transport_url not found in DEFAULT section\"\n exit 2026-01-21 11:38:24.332069 | controller | 1\n fi\n echo \"Found RPC transport_url: $rpc_transport_url\"\n \n 2026-01-21 11:38:24.332075 | controller | \ # Verify the RPC transport_url contains the correct vhost (watcher-rpc)\n 2026-01-21 11:38:24.332080 | controller | \ if ! echo \"$rpc_transport_url\" | grep -q '/watcher-rpc'; then\n echo 2026-01-21 11:38:24.332086 | controller | \"RPC transport_url does not contain expected vhost '/watcher-rpc'\"\n exit 2026-01-21 11:38:24.332092 | controller | 1\n fi\n echo \"Successfully verified vhost 'watcher-rpc' in RPC transport_url\"\n 2026-01-21 11:38:24.332098 | controller | \ \n # Verify the RPC transport_url contains the correct username (watcher-rpc)\n 2026-01-21 11:38:24.332104 | controller | \ if ! echo \"$rpc_transport_url\" | grep -q 'watcher-rpc:'; then\n echo 2026-01-21 11:38:24.332110 | controller | \"RPC transport_url does not contain expected username 'watcher-rpc:'\"\n exit 2026-01-21 11:38:24.332115 | controller | 1\n fi\n echo \"Successfully verified username 'watcher-rpc' in RPC 2026-01-21 11:38:24.332121 | controller | transport_url\"\n \n # Verify oslo_messaging_notifications section 2026-01-21 11:38:24.332127 | controller | has transport_url configured\n notif_transport_url=$(kubectl exec -n $NAMESPACE 2026-01-21 11:38:24.332133 | controller | ${WATCHER_API_POD} -c watcher-api -- cat /etc/watcher/watcher.conf.d/00-default.conf 2026-01-21 11:38:24.332139 | controller | | grep -A 5 '\\[oslo_messaging_notifications\\]' | grep 'transport_url' || true)\n 2026-01-21 11:38:24.332145 | controller | \ if [ -z \"$notif_transport_url\" ]; then\n echo \"transport_url 2026-01-21 11:38:24.332153 | controller | not found in oslo_messaging_notifications section\"\n exit 1\n fi\n 2026-01-21 11:38:24.332159 | controller | \ echo \"Found notifications transport_url: $notif_transport_url\"\n \n 2026-01-21 11:38:24.332165 | controller | \ # Verify the notifications transport_url contains the correct vhost (watcher-notifications)\n 2026-01-21 11:38:24.332171 | controller | \ if ! echo \"$notif_transport_url\" | grep -q '/watcher-notifications'; then\n 2026-01-21 11:38:24.332177 | controller | \ echo \"Notifications transport_url does not contain expected vhost '/watcher-notifications'\"\n 2026-01-21 11:38:24.332183 | controller | \ exit 1\n fi\n echo \"Successfully verified vhost 'watcher-notifications' 2026-01-21 11:38:24.332189 | controller | in notifications transport_url\"\n \n # Verify the notifications transport_url 2026-01-21 11:38:24.332194 | controller | contains the correct username (watcher-notifications)\n if ! echo \"$notif_transport_url\" 2026-01-21 11:38:24.332200 | controller | | grep -q 'watcher-notifications:'; then\n echo \"Notifications transport_url 2026-01-21 11:38:24.332206 | controller | does not contain expected username 'watcher-notifications:'\"\n exit 1\n 2026-01-21 11:38:24.332212 | controller | \ fi\n echo \"Successfully verified username 'watcher-notifications' 2026-01-21 11:38:24.332227 | controller | in notifications transport_url\"\n \n exit 0\n ]\n logger.go:42: 2026-01-21 11:38:24.332237 | controller | 11:26:17 | watcher-rmquser/1-deploy | + kubectl wait --for=condition=Ready watcher/watcher-kuttl 2026-01-21 11:38:24.332246 | controller | -n watcher-kuttl-default --timeout=300s\n logger.go:42: 11:31:18 | watcher-rmquser/1-deploy 2026-01-21 11:38:24.332259 | controller | | error: timed out waiting for the condition on watchers/watcher-kuttl\n logger.go:42: 2026-01-21 11:38:24.332269 | controller | 11:31:18 | watcher-rmquser/1-deploy | test step failed 1-deploy\n case.go:396: 2026-01-21 11:38:24.332277 | controller | failed in step 1-deploy\n case.go:398: transporturls.rabbitmq.openstack.org \"watcher-kuttl-watcher-transport\" 2026-01-21 11:38:24.332285 | controller | not found\n case.go:398: transporturls.rabbitmq.openstack.org \"watcher-kuttl-watcher-notification-rabbitmq-notifications\" 2026-01-21 11:38:24.332291 | controller | not found\n case.go:398: command \"kubectl wait --for=condition=Ready watcher/watcher-kuttl 2026-01-21 11:38:24.332297 | controller | -n $NAMESP...\" exceeded 300 sec timeout, context deadline exceeded\n logger.go:42: 2026-01-21 11:38:24.332303 | controller | 11:31:18 | watcher-rmquser | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-notification\n 2026-01-21 11:38:24.332309 | controller | \ logger.go:42: 11:31:18 | watcher-notification | Skipping creation of user-supplied 2026-01-21 11:38:24.332315 | controller | namespace: watcher-kuttl-default\n logger.go:42: 11:31:18 | watcher-notification/0-cleanup-watcher 2026-01-21 11:38:24.332321 | controller | | starting test step 0-cleanup-watcher\n logger.go:42: 11:31:18 | watcher-notification/0-cleanup-watcher 2026-01-21 11:38:24.332327 | controller | | test step completed 0-cleanup-watcher\n logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332333 | controller | | starting test step 1-deploy-with-notification\n logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332339 | controller | | Watcher:watcher-kuttl-default/watcher-kuttl created\n logger.go:42: 11:31:18 2026-01-21 11:38:24.332345 | controller | | watcher-notification/1-deploy-with-notification | running command: [sh -c set 2026-01-21 11:38:24.332350 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.332356 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.332362 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.332368 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.332374 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.332380 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.332386 | controller | 11:31:18 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332392 | controller | \ logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.332398 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.332404 | controller | \ logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332410 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.332415 | controller | \ logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332421 | controller | head -1\n logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332427 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332433 | controller | | + APIPOD=\n logger.go:42: 11:31:19 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332439 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332445 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.332451 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.332461 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.332467 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.332473 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.332479 | controller | 11:31:19 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332484 | controller | \ logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.332490 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.332505 | controller | \ logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332518 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.332527 | controller | \ logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332535 | controller | head -1\n logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332544 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332551 | controller | | + APIPOD=\n logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332557 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332566 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.332572 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.332578 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.332584 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.332590 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.332599 | controller | 11:31:21 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332605 | controller | \ logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.332611 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.332617 | controller | \ logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332623 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.332629 | controller | \ logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332635 | controller | head -1\n logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332641 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332647 | controller | | + APIPOD=\n logger.go:42: 11:31:22 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332653 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332659 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.332665 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.332671 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.332677 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.332683 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.332693 | controller | 11:31:22 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332699 | controller | \ logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.332705 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.332711 | controller | \ logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332743 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.332752 | controller | \ logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332759 | controller | grep -v '^$'\n logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332766 | controller | | ++ head -1\n logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332772 | controller | | + APIPOD=\n logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332778 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332784 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.332790 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.332796 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.332802 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.332808 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.332814 | controller | 11:31:24 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332828 | controller | \ logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.332837 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.332845 | controller | \ logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332852 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.332860 | controller | \ logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332868 | controller | grep -v '^$'\n logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332875 | controller | | ++ head -1\n logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332883 | controller | | + APIPOD=\n logger.go:42: 11:31:25 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.332891 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332904 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.332918 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.332925 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.332933 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.332941 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.332949 | controller | 11:31:25 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.332957 | controller | \ logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.332965 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.332979 | controller | \ logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.332987 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.332999 | controller | \ logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333005 | controller | grep -v '^$'\n logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333011 | controller | | ++ head -1\n logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333017 | controller | | + APIPOD=\n logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333022 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333028 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333034 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333040 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333046 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333052 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333058 | controller | 11:31:27 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333064 | controller | \ logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333069 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333075 | controller | \ logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333084 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333090 | controller | \ logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333096 | controller | head -1\n logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333102 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333108 | controller | | + APIPOD=\n logger.go:42: 11:31:28 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333113 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333119 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333125 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333131 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333137 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333154 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333170 | controller | 11:31:28 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333185 | controller | \ logger.go:42: 11:31:28 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333193 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333201 | controller | \ logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333207 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333213 | controller | \ logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333224 | controller | head -1\n logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333230 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333236 | controller | | + APIPOD=\n logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333242 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333248 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333254 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333260 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333266 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333271 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333277 | controller | 11:31:30 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333283 | controller | \ logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333289 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333295 | controller | \ logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333301 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333307 | controller | \ logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333313 | controller | grep -v '^$'\n logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333319 | controller | | ++ head -1\n logger.go:42: 11:31:31 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333324 | controller | | + APIPOD=\n logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333330 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333336 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333342 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333348 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333354 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333360 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333366 | controller | 11:31:32 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333375 | controller | \ logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333381 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333387 | controller | \ logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333393 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333399 | controller | \ logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333405 | controller | head -1\n logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333411 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333416 | controller | | + APIPOD=\n logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333425 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333431 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333437 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333454 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333482 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333491 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333498 | controller | 11:31:33 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333504 | controller | \ logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333510 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333516 | controller | \ logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333522 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333534 | controller | \ logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333549 | controller | grep -v '^$'\n logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333557 | controller | | ++ head -1\n logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333563 | controller | | + APIPOD=\n logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333569 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333575 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333581 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333586 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333592 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333609 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333615 | controller | 11:31:35 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333621 | controller | \ logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333627 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333632 | controller | \ logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333638 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333644 | controller | \ logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333650 | controller | head -1\n logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333661 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333667 | controller | | + APIPOD=\n logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333673 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333679 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333684 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333697 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333703 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333709 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333733 | controller | 11:31:36 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333739 | controller | \ logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333745 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333751 | controller | \ logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333757 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333763 | controller | \ logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333768 | controller | grep -v '^$'\n logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333774 | controller | | ++ head -1\n logger.go:42: 11:31:37 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333780 | controller | | + APIPOD=\n logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333800 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333816 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333829 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333837 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333845 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333852 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333863 | controller | 11:31:38 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333878 | controller | \ logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.333886 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.333894 | controller | \ logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333902 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.333909 | controller | \ logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.333916 | controller | grep -v '^$'\n logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333923 | controller | | ++ head -1\n logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333930 | controller | | + APIPOD=\n logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.333937 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333945 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.333951 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.333958 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.333965 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.333980 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.333987 | controller | 11:31:39 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.333994 | controller | \ logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334001 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334008 | controller | \ logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334015 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334022 | controller | \ logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334029 | controller | head -1\n logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334036 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:40 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334043 | controller | | + APIPOD=\n logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334050 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334058 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.334065 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.334071 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.334079 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334085 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334093 | controller | 11:31:41 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334100 | controller | \ logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334107 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334114 | controller | \ logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334121 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334128 | controller | \ logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334134 | controller | grep -v '^$'\n logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334159 | controller | | ++ head -1\n logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334168 | controller | | + APIPOD=\n logger.go:42: 11:31:42 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334175 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334183 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.334190 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.334198 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.334206 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334213 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334220 | controller | 11:31:42 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334228 | controller | \ logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334241 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334249 | controller | \ logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334257 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334280 | controller | \ logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334295 | controller | head -1\n logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334308 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334325 | controller | | + APIPOD=\n logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334335 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334343 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.334350 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.334357 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.334365 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334372 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334379 | controller | 11:31:44 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334386 | controller | \ logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334394 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334401 | controller | \ logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334408 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334415 | controller | \ logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334422 | controller | head -1\n logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334429 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334436 | controller | | + APIPOD=\n logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334444 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334452 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.334459 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.334466 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.334472 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334480 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334486 | controller | 11:31:46 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334493 | controller | \ logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334501 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334508 | controller | \ logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334516 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334545 | controller | \ logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334616 | controller | head -1\n logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334635 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334645 | controller | | + APIPOD=\n logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334653 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334659 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.334665 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.334671 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.334677 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334683 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334689 | controller | 11:31:47 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334695 | controller | \ logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334701 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334707 | controller | \ logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334728 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334739 | controller | \ logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334745 | controller | head -1\n logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334751 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:48 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334757 | controller | | + APIPOD=\n logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334763 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334769 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.334780 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.334786 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.334792 | controller | |grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334798 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334804 | controller | 11:31:49 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334810 | controller | \ logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334816 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334822 | controller | \ logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334827 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.334833 | controller | \ logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334839 | controller | grep -v '^$'\n logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334845 | controller | | ++ head -1\n logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334856 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334862 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.334868 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.334874 | controller | \ logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.334880 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.334885 | controller | \ logger.go:42: 11:31:50 | watcher-notification/1-deploy-with-notification | error: 2026-01-21 11:38:24.334891 | controller | unable to upgrade connection: container not found (\"watcher-api\")\n logger.go:42: 2026-01-21 11:38:24.334897 | controller | 11:31:50 | watcher-notification/1-deploy-with-notification | ++ echo\n logger.go:42: 2026-01-21 11:38:24.334903 | controller | 11:31:50 | watcher-notification/1-deploy-with-notification | + '[' 0 == 1 ']'\n 2026-01-21 11:38:24.334909 | controller | \ logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.334926 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.334935 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.334943 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.334950 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.334958 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.334967 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.334974 | controller | 11:31:51 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.334980 | controller | \ logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.334986 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.334992 | controller | \ logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.334997 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.335003 | controller | \ logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335009 | controller | head -1\n logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335019 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335025 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335031 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335037 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.335043 | controller | \ logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.335049 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.335054 | controller | \ logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | error: 2026-01-21 11:38:24.335060 | controller | unable to upgrade connection: container not found (\"watcher-api\")\n logger.go:42: 2026-01-21 11:38:24.335066 | controller | 11:31:52 | watcher-notification/1-deploy-with-notification | ++ echo\n logger.go:42: 2026-01-21 11:38:24.335076 | controller | 11:31:52 | watcher-notification/1-deploy-with-notification | + '[' 0 == 1 ']'\n 2026-01-21 11:38:24.335082 | controller | \ logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.335087 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.335093 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.335099 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.335105 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.335111 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.335117 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.335123 | controller | 11:31:53 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.335129 | controller | \ logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.335135 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.335143 | controller | \ logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335149 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.335155 | controller | \ logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335161 | controller | head -1\n logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335167 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335173 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335182 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335190 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.335196 | controller | \ logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.335210 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.335220 | controller | \ logger.go:42: 11:31:54 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335228 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.335236 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.335244 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.335253 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.335259 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.335265 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.335271 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.335277 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.335287 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.335293 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.335298 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.335304 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.335311 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.335316 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.335322 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.335328 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.335334 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.335340 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.335346 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.335352 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.335357 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.335363 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.335369 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.335375 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.335381 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.335387 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.335393 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.335399 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.335405 | controller | 11:31:54 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.335411 | controller | \ logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.335416 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.335422 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.335428 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.335434 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.335440 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.335446 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.335460 | controller | 11:31:55 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.335469 | controller | \ logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.335477 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.335484 | controller | \ logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335498 | controller | head -1\n logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335505 | controller | | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.335511 | controller | \ logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335517 | controller | grep -v '^$'\n logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335523 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335529 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335534 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.335540 | controller | \ logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.335546 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.335552 | controller | \ logger.go:42: 11:31:56 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335558 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.335564 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.335569 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.335575 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.335581 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.335587 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.335593 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.335599 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.335605 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.335611 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.335623 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.335629 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.335638 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.335644 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.335650 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.335656 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.335662 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.335667 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.335673 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.335679 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.335689 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.335694 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.335700 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.335706 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.335735 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.335748 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.335757 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.335765 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.335773 | controller | 11:31:56 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.335780 | controller | \ logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.335785 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.335791 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.335797 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.335803 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.335809 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.335815 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.335820 | controller | 11:31:57 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.335826 | controller | \ logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.335832 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.335838 | controller | \ logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335844 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.335850 | controller | \ logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335855 | controller | grep -v '^$'\n logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335861 | controller | | ++ head -1\n logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335867 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335873 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.335879 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.335891 | controller | \ logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.335897 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.335903 | controller | \ logger.go:42: 11:31:58 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.335909 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.335918 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.335924 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.335930 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.335936 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.335942 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.335948 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.335954 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.335959 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.335965 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.335971 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.335977 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.335983 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.335998 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.336006 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.336014 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.336022 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.336030 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.336038 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.336044 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.336049 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.336055 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.336061 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.336067 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.336073 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.336079 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.336085 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.336091 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.336097 | controller | 11:31:58 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.336106 | controller | \ logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.336112 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.336118 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.336128 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.336134 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.336140 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.336149 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.336154 | controller | 11:31:59 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.336160 | controller | \ logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.336166 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.336172 | controller | \ logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336178 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.336184 | controller | \ logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336189 | controller | head -1\n logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336198 | controller | | ++ grep -v '^$'\n logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336204 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336210 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336216 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.336221 | controller | \ logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.336228 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.336235 | controller | \ logger.go:42: 11:32:00 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336241 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.336248 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.336254 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.336270 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.336280 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.336289 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.336296 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.336304 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.336312 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.336319 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.336325 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.336331 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.336341 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.336347 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.336353 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.336359 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.336365 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.336371 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.336377 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.336383 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.336389 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.336394 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.336400 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.336406 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.336412 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.336418 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.336424 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.336430 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.336436 | controller | 11:32:00 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.336441 | controller | \ logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.336447 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.336453 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.336459 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.336465 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.336471 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.336477 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.336483 | controller | 11:32:01 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.336488 | controller | \ logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.336494 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.336500 | controller | \ logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336506 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.336512 | controller | \ logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336521 | controller | head -1\n logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336536 | controller | | ++ grep -v '^$'\n logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336549 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336557 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336564 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.336572 | controller | \ logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.336579 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.336585 | controller | \ logger.go:42: 11:32:02 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336591 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.336597 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.336603 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.336609 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.336615 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.336621 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.336627 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.336633 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.336639 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.336645 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.336650 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.336656 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.336662 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.336668 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.336674 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.336680 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.336686 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.336692 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.336698 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.336704 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.336710 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.336733 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.336739 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.336749 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.336755 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.336761 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.336766 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.336772 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.336782 | controller | 11:32:02 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.336788 | controller | \ logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.336794 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.336809 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.336822 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.336831 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.336839 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.336848 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.336855 | controller | 11:32:03 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.336861 | controller | \ logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.336867 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.336873 | controller | \ logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336879 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.336885 | controller | \ logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336891 | controller | grep -v '^$'\n logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336897 | controller | | ++ head -1\n logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336902 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336908 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.336914 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.336929 | controller | \ logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.336935 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.336941 | controller | \ logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.336947 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.336952 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.336958 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.336964 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.336974 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.336980 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.336986 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.336992 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.336998 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.337003 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.337009 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.337015 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.337021 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.337027 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.337033 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.337039 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.337045 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.337051 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.337057 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.337075 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.337083 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.337091 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.337099 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.337107 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.337115 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.337122 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.337128 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.337134 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.337140 | controller | 11:32:03 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.337146 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | running 2026-01-21 11:38:24.337152 | controller | command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc 2026-01-21 11:38:24.337158 | controller | get pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.337164 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.337170 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) |grep 2026-01-21 11:38:24.337176 | controller | -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+=') 2026-01-21 11:38:24.337186 | controller | == 1 ]\n else\n exit 1\n fi\n ]\n logger.go:42: 2026-01-21 11:38:24.337192 | controller | 11:32:05 | watcher-notification/1-deploy-with-notification | + oc project watcher-kuttl-default\n 2026-01-21 11:38:24.337198 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | Already 2026-01-21 11:38:24.337203 | controller | on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.337209 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.337215 | controller | oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.337221 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.337227 | controller | grep -v '^$'\n logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.337233 | controller | | ++ head -1\n logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.337239 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.337245 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification 2026-01-21 11:38:24.337251 | controller | | ++ grep -czPo '\\[oslo_messaging_notifications\\]\\s+driver\\s+=\\s+messagingv2\\s+transport_url\\s+='\n 2026-01-21 11:38:24.337256 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | +++ 2026-01-21 11:38:24.337262 | controller | oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.337268 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | ++ 2026-01-21 11:38:24.337274 | controller | echo '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.337280 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.337286 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.337292 | controller | = '[database]' connection = 'mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.337298 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.337304 | controller | driver = messagingv2 transport_url = 'rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.337310 | controller | '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.337316 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.337322 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.337337 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.337345 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.337353 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.337361 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.337374 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.337381 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.337393 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.337399 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.337405 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.337410 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.337416 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.337422 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.337428 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.337434 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt '[cinder_client]' endpoint_type 2026-01-21 11:38:24.337440 | controller | = internal '[glance_client]' endpoint_type = internal '[ironic_client]' endpoint_type 2026-01-21 11:38:24.337446 | controller | = internal '[keystone_client]' interface = internal '[neutron_client]' endpoint_type 2026-01-21 11:38:24.337452 | controller | = internal '[nova_client]' endpoint_type = internal '[placement_client]' interface 2026-01-21 11:38:24.337457 | controller | = internal '[watcher_cluster_data_model_collectors.compute]' period = 900 '[watcher_cluster_data_model_collectors.baremetal]' 2026-01-21 11:38:24.337463 | controller | period = 900 '[watcher_cluster_data_model_collectors.storage]' period = 900\n logger.go:42: 2026-01-21 11:38:24.337469 | controller | 11:32:05 | watcher-notification/1-deploy-with-notification | + '[' 1 == 1 ']'\n 2026-01-21 11:38:24.337475 | controller | \ logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | test 2026-01-21 11:38:24.337481 | controller | step completed 1-deploy-with-notification\n logger.go:42: 11:32:05 | watcher-notification/2-cleanup-watcher 2026-01-21 11:38:24.337487 | controller | | starting test step 2-cleanup-watcher\n logger.go:42: 11:32:16 | watcher-notification/2-cleanup-watcher 2026-01-21 11:38:24.337493 | controller | | test step completed 2-cleanup-watcher\n logger.go:42: 11:32:16 | watcher-notification 2026-01-21 11:38:24.337499 | controller | | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-cinder\n logger.go:42: 2026-01-21 11:38:24.337505 | controller | 11:32:16 | watcher-cinder | Skipping creation of user-supplied namespace: watcher-kuttl-default\n 2026-01-21 11:38:24.337510 | controller | \ logger.go:42: 11:32:16 | watcher-cinder/0-cleanup-watcher | starting test step 2026-01-21 11:38:24.337516 | controller | 0-cleanup-watcher\n logger.go:42: 11:32:16 | watcher-cinder/0-cleanup-watcher 2026-01-21 11:38:24.337522 | controller | | test step completed 0-cleanup-watcher\n logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337528 | controller | | starting test step 1-deploy-watcher-no-cinder\n logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337537 | controller | | Watcher:watcher-kuttl-default/watcher-kuttl created\n logger.go:42: 11:32:16 2026-01-21 11:38:24.337543 | controller | | watcher-cinder/1-deploy-watcher-no-cinder | running command: [sh -c set -euxo 2026-01-21 11:38:24.337549 | controller | pipefail\n # check that the decision engine correctly detects that there 2026-01-21 11:38:24.337555 | controller | is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.337561 | controller | | grep -c 'Block storage service is not enabled, skipping storage collector')\" 2026-01-21 11:38:24.337567 | controller | == 2 ]\n ]\n logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337573 | controller | | ++ oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.337578 | controller | 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep -c 'Block storage 2026-01-21 11:38:24.337588 | controller | service is not enabled, skipping storage collector'\n logger.go:42: 11:32:16 2026-01-21 11:38:24.337594 | controller | | watcher-cinder/1-deploy-watcher-no-cinder | error: error from server (NotFound): 2026-01-21 11:38:24.337600 | controller | pods \"watcher-kuttl-decision-engine-0\" not found in namespace \"watcher-kuttl-default\"\n 2026-01-21 11:38:24.337615 | controller | \ logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 0 2026-01-21 11:38:24.337623 | controller | == 2 ']'\n logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337631 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.337639 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.337647 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.337655 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.337662 | controller | 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.337668 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337674 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.337680 | controller | \ logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.337686 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.337692 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337698 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337704 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.337710 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.337733 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.337739 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.337745 | controller | 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.337751 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337757 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.337763 | controller | \ logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.337769 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.337774 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337780 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337786 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.337792 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.337798 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.337804 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.337810 | controller | 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.337816 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337826 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.337832 | controller | \ logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.337838 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.337843 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337849 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337859 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.337866 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.337872 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.337878 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.337884 | controller | 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.337890 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337895 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.337901 | controller | \ logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.337907 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.337921 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337930 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:22 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337938 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.337946 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.337954 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.337961 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.337969 | controller | 11:32:23 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.337975 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:23 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.337981 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.337987 | controller | \ logger.go:42: 11:32:24 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.337993 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.337999 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:24 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338005 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338011 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338017 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338023 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338028 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338034 | controller | 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338040 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338046 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338056 | controller | \ logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338062 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338068 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338077 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338083 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338089 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338095 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338101 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338107 | controller | 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338113 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338119 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338125 | controller | \ logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338130 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338136 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338142 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338148 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338154 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338160 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338169 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338175 | controller | 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338183 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338189 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338195 | controller | \ logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338201 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338215 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338224 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:28 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338232 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338240 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338247 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338255 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338262 | controller | 11:32:28 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338269 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:28 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338276 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338286 | controller | \ logger.go:42: 11:32:29 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338292 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338298 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:29 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338304 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338310 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338316 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338322 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338327 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338333 | controller | 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338339 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338345 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338351 | controller | \ logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338357 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338362 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338368 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338374 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338380 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338386 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338391 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338397 | controller | 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338403 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338409 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338415 | controller | \ logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338421 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338426 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338432 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338438 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338444 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338450 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338456 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338462 | controller | 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338467 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338473 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338479 | controller | \ logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338495 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338511 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338520 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338554 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338584 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338592 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338600 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338607 | controller | 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338614 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338620 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338626 | controller | \ logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338631 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338637 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338643 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338649 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338655 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338661 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338667 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338672 | controller | 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338678 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338684 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338690 | controller | \ logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338696 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338702 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338707 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338759 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338828 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338846 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338853 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338860 | controller | 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338868 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338875 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338882 | controller | \ logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.338896 | controller | error from server (NotFound): pods \"watcher-kuttl-decision-engine-0\" not found 2026-01-21 11:38:24.338903 | controller | in namespace \"watcher-kuttl-default\"\n logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338910 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:37 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338917 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.338924 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.338931 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.338938 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.338950 | controller | 11:32:37 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.338958 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:37 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.338965 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.338972 | controller | \ logger.go:42: 11:32:38 | watcher-cinder/1-deploy-watcher-no-cinder | Error from 2026-01-21 11:38:24.338997 | controller | server (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.339005 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:32:38 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339012 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339019 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339026 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339033 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339044 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339051 | controller | 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339058 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339065 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339072 | controller | \ logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 0 2026-01-21 11:38:24.339079 | controller | == 2 ']'\n logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339085 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339092 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339099 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339106 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339113 | controller | 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339120 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339127 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339134 | controller | \ logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 0 2026-01-21 11:38:24.339141 | controller | == 2 ']'\n logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339148 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339155 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339166 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339173 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339180 | controller | 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339187 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339194 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339201 | controller | \ logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 0 2026-01-21 11:38:24.339208 | controller | == 2 ']'\n logger.go:42: 11:32:42 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339215 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339222 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339230 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339238 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339245 | controller | 11:32:42 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339253 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:42 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339261 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339269 | controller | \ logger.go:42: 11:32:43 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 1 2026-01-21 11:38:24.339277 | controller | == 2 ']'\n logger.go:42: 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339286 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339293 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339301 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339308 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339315 | controller | 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep -c 'Block storage 2026-01-21 11:38:24.339322 | controller | service is not enabled, skipping storage collector'\n logger.go:42: 11:32:44 2026-01-21 11:38:24.339342 | controller | | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339350 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339357 | controller | | + '[' 2 == 2 ']'\n logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339364 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339371 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339378 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339386 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339393 | controller | 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339401 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339408 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339417 | controller | \ logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 2 2026-01-21 11:38:24.339424 | controller | == 2 ']'\n logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339436 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339443 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339450 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339457 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339465 | controller | 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339472 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339479 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339486 | controller | \ logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 2 2026-01-21 11:38:24.339494 | controller | == 2 ']'\n logger.go:42: 11:32:47 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339503 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339510 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339518 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339525 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339533 | controller | 11:32:47 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339540 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:47 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339548 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339555 | controller | \ logger.go:42: 11:32:48 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 2 2026-01-21 11:38:24.339562 | controller | == 2 ']'\n logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339570 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339577 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.339584 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c 'Block storage service is 2026-01-21 11:38:24.339591 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.339598 | controller | 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339605 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339614 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.339621 | controller | \ logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | + '[' 2 2026-01-21 11:38:24.339629 | controller | == 2 ']'\n logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder 2026-01-21 11:38:24.339636 | controller | | test step completed 1-deploy-watcher-no-cinder\n logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339644 | controller | | starting test step 2-deploy-cinder\n logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339652 | controller | | OpenStackControlPlane:watcher-kuttl-default/openstack updated\n logger.go:42: 2026-01-21 11:38:24.339660 | controller | 11:32:49 | watcher-cinder/2-deploy-cinder | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.339674 | controller | \ # check that the decision detects that there is a cinder service and\n # 2026-01-21 11:38:24.339681 | controller | does not log that storage collector is skipped\n [ \"$(oc logs -n $NAMESPACE 2026-01-21 11:38:24.339705 | controller | watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is not enabled, 2026-01-21 11:38:24.339742 | controller | skipping storage collector')\" == 0 ]\n ]\n logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339758 | controller | | ++ oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.339765 | controller | 11:32:49 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.339773 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339780 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339787 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339795 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.339802 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.339809 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.339817 | controller | 0 ]\n ]\n logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.339824 | controller | grep -c 'Block storage service is not enabled, skipping storage collector'\n logger.go:42: 2026-01-21 11:38:24.339832 | controller | 11:32:50 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.339840 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339847 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:51 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339855 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339863 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.339871 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.339878 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.339886 | controller | 0 ]\n ]\n logger.go:42: 11:32:51 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.339893 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.339901 | controller | 11:32:51 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.339913 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:52 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339921 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339928 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.339936 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.339944 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.339952 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.339960 | controller | 0 ]\n ]\n logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.339967 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.339975 | controller | 11:32:53 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.339982 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339989 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.339997 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340004 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340017 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340025 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340033 | controller | 0 ]\n ]\n logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340042 | controller | grep -c 'Block storage service is not enabled, skipping storage collector'\n logger.go:42: 2026-01-21 11:38:24.340049 | controller | 11:32:54 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.340057 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340070 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340078 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340085 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340093 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340115 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340125 | controller | 0 ]\n ]\n logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340133 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340140 | controller | 11:32:55 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340148 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340156 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340164 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340172 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340180 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340189 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340197 | controller | 0 ]\n ]\n logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340204 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340211 | controller | 11:32:57 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340218 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340226 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340233 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340241 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340249 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340257 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340264 | controller | 0 ]\n ]\n logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340271 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340285 | controller | 11:32:58 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340293 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340300 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340313 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340321 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340329 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340337 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340344 | controller | 0 ]\n ]\n logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340351 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340359 | controller | 11:32:59 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340366 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340374 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:00 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340381 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340389 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340396 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340411 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340419 | controller | 0 ]\n ]\n logger.go:42: 11:33:00 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340427 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340435 | controller | 11:33:00 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340442 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:01 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340449 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340457 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340464 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340472 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340494 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340503 | controller | 0 ]\n ]\n logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340509 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340516 | controller | 11:33:02 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340523 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340531 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340539 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340547 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340555 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340562 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340569 | controller | 0 ]\n ]\n logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340577 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340584 | controller | 11:33:03 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340599 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340606 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340613 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340621 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340628 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340635 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340642 | controller | 0 ]\n ]\n logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340650 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340658 | controller | 11:33:04 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340666 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340673 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:05 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340680 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340687 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340695 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340701 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340707 | controller | 0 ]\n ]\n logger.go:42: 11:33:05 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340735 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340746 | controller | 11:33:05 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340752 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:06 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340758 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340764 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340770 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340775 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340781 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340787 | controller | 0 ]\n ]\n logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340793 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340799 | controller | 11:33:07 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340805 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340811 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340816 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340822 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340828 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340846 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340853 | controller | 0 ]\n ]\n logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340858 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340869 | controller | 11:33:08 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340875 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340881 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:09 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340886 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340892 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340901 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340907 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340913 | controller | 0 ]\n ]\n logger.go:42: 11:33:09 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340919 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340925 | controller | 11:33:09 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340930 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:10 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340936 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.340942 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.340948 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.340954 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.340961 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.340969 | controller | 0 ]\n ]\n logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.340976 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.340984 | controller | 11:33:11 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.340992 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341000 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341008 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341020 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341028 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341035 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341042 | controller | 0 ]\n ]\n logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341054 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341068 | controller | 11:33:12 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341076 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341083 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:13 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341089 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341096 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341104 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341112 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341125 | controller | 0 ]\n ]\n logger.go:42: 11:33:13 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341133 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341141 | controller | 11:33:13 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341148 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:14 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341155 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341163 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341169 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341177 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341184 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341208 | controller | 0 ]\n ]\n logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341218 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341225 | controller | 11:33:15 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341234 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341242 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341250 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341259 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341268 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341276 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341286 | controller | 0 ]\n ]\n logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341294 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341302 | controller | 11:33:16 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341309 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341317 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341325 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341332 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341340 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341348 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341356 | controller | 0 ]\n ]\n logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341370 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341378 | controller | 11:33:17 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341385 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341392 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341399 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341406 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341416 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341422 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341428 | controller | 0 ]\n ]\n logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341434 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341440 | controller | 11:33:19 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341446 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341452 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341458 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341463 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341469 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341475 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341481 | controller | 0 ]\n ]\n logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341487 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341493 | controller | 11:33:20 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341500 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341507 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341515 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341523 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341531 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341539 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341560 | controller | 0 ]\n ]\n logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341568 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341574 | controller | 11:33:21 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341580 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341586 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:22 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341592 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341598 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341604 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341610 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341615 | controller | 0 ]\n ]\n logger.go:42: 11:33:22 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341621 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341627 | controller | 11:33:22 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341633 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:23 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341639 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341655 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341671 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341682 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341690 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341697 | controller | 0 ]\n ]\n logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341704 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341731 | controller | 11:33:24 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341744 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341752 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341758 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341765 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341772 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341779 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341786 | controller | 0 ]\n ]\n logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341793 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341800 | controller | 11:33:25 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341807 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341814 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:26 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341822 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341830 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341837 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341843 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341849 | controller | 0 ]\n ]\n logger.go:42: 11:33:26 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341855 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341861 | controller | 11:33:26 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341866 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:27 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341872 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341878 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341884 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341895 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341901 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341920 | controller | 0 ]\n ]\n logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341927 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341933 | controller | 11:33:28 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.341939 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341950 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.341956 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.341962 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.341967 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.341973 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.341982 | controller | 0 ]\n ]\n logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.341988 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.341994 | controller | 11:33:29 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342000 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342006 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:30 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342012 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342018 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342024 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342030 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342035 | controller | 0 ]\n ]\n logger.go:42: 11:33:30 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342041 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342047 | controller | 11:33:30 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342053 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:31 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342059 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342065 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342071 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342077 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342082 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342088 | controller | 0 ]\n ]\n logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342094 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342100 | controller | 11:33:32 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342106 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342112 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342118 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342124 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342129 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342135 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342141 | controller | 0 ]\n ]\n logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342147 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342156 | controller | 11:33:33 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342163 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342171 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342179 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342188 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342365 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342385 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342408 | controller | 0 ]\n ]\n logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342417 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342424 | controller | 11:33:34 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342432 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342447 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:35 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342453 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342459 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342465 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342471 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342477 | controller | 0 ]\n ]\n logger.go:42: 11:33:35 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342483 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342489 | controller | 11:33:35 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342495 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:36 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342502 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342508 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342514 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342519 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342525 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342531 | controller | 0 ]\n ]\n logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342537 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342543 | controller | 11:33:37 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342551 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342587 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342596 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342603 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342611 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342618 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342630 | controller | 0 ]\n ]\n logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342638 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342646 | controller | 11:33:38 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342653 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342659 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342665 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342671 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342677 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342683 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342689 | controller | 0 ]\n ]\n logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342694 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342700 | controller | 11:33:39 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342706 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342727 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342737 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342743 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342749 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342759 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342765 | controller | 0 ]\n ]\n logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342782 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342789 | controller | 11:33:41 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342795 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342800 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342807 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342813 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342819 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342824 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342830 | controller | 0 ]\n ]\n logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342836 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342842 | controller | 11:33:42 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342848 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342854 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342859 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342865 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342871 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342881 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342887 | controller | 0 ]\n ]\n logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342892 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342898 | controller | 11:33:43 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342904 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342910 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342916 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342922 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342928 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.342934 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.342939 | controller | 0 ]\n ]\n logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.342945 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.342952 | controller | 11:33:44 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.342959 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342967 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:45 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.342975 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.342982 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.342993 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343001 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343008 | controller | 0 ]\n ]\n logger.go:42: 11:33:45 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343015 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343026 | controller | 11:33:45 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343034 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:46 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343042 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343051 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343059 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343068 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343076 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343084 | controller | 0 ]\n ]\n logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343107 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343117 | controller | 11:33:47 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343125 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343134 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343142 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343155 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343164 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343172 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343181 | controller | 0 ]\n ]\n logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343189 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343197 | controller | 11:33:48 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343206 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343214 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343222 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343232 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343242 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343250 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343260 | controller | 0 ]\n ]\n logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343270 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343279 | controller | 11:33:49 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343289 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343297 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:50 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343306 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343314 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343322 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343329 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343337 | controller | 0 ]\n ]\n logger.go:42: 11:33:50 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343344 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343351 | controller | 11:33:50 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343358 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:51 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343366 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343375 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343387 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343396 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343405 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343413 | controller | 0 ]\n ]\n logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343421 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343430 | controller | 11:33:52 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343438 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343452 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343461 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343468 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343476 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343484 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343491 | controller | 0 ]\n ]\n logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343513 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343531 | controller | 11:33:53 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343541 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343548 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:54 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343554 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343560 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343566 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343572 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343578 | controller | 0 ]\n ]\n logger.go:42: 11:33:54 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343584 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343590 | controller | 11:33:54 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343595 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:55 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343601 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343607 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343613 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343619 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343625 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343631 | controller | 0 ]\n ]\n logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343637 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343642 | controller | 11:33:56 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343648 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343654 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343660 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343666 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343672 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343678 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343684 | controller | 0 ]\n ]\n logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343695 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343701 | controller | 11:33:57 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343729 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343739 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:33:58 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343745 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343751 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343757 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343763 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343769 | controller | 0 ]\n ]\n logger.go:42: 11:33:58 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343775 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343781 | controller | 11:33:58 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343787 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:33:59 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343794 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343800 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343806 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343812 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343817 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343823 | controller | 0 ]\n ]\n logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343830 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343850 | controller | 11:34:00 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343857 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343863 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:01 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343869 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343875 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343881 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343887 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343893 | controller | 0 ]\n ]\n logger.go:42: 11:34:01 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343899 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343905 | controller | 11:34:01 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343911 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:02 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343917 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343923 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.343929 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.343935 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.343941 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.343948 | controller | 0 ]\n ]\n logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.343967 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.343981 | controller | 11:34:03 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.343988 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.343996 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344009 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344016 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344027 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344035 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344042 | controller | 0 ]\n ]\n logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344050 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344058 | controller | 11:34:04 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344065 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344073 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:05 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344080 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344088 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344095 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344103 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344110 | controller | 0 ]\n ]\n logger.go:42: 11:34:05 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344118 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344125 | controller | 11:34:05 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344133 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:06 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344141 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344148 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344156 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344162 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344168 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344174 | controller | 0 ]\n ]\n logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344182 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344200 | controller | 11:34:07 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344207 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344212 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:08 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344218 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344224 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344230 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344241 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344247 | controller | 0 ]\n ]\n logger.go:42: 11:34:08 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344253 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344259 | controller | 11:34:08 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344265 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:09 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344270 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344276 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344282 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344288 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344294 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344306 | controller | 0 ]\n ]\n logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344313 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344326 | controller | 11:34:10 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344340 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344348 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344355 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344363 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344371 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344378 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344384 | controller | 0 ]\n ]\n logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344390 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344395 | controller | 11:34:11 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344401 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344408 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:12 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344423 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344436 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344444 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344451 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344458 | controller | 0 ]\n ]\n logger.go:42: 11:34:12 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344466 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344473 | controller | 11:34:12 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344480 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:13 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344487 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344495 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344509 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344517 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344524 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344531 | controller | 0 ]\n ]\n logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344538 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344561 | controller | 11:34:14 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344569 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344576 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344584 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344591 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344598 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344604 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344611 | controller | 0 ]\n ]\n logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344618 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344625 | controller | 11:34:15 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344632 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344644 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344651 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344658 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344665 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344672 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344678 | controller | 0 ]\n ]\n logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344685 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344692 | controller | 11:34:17 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344699 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344706 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344745 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344753 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344760 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344767 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344774 | controller | 0 ]\n ]\n logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344781 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344788 | controller | 11:34:18 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344795 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344802 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:19 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344813 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344820 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344827 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344834 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344841 | controller | 0 ]\n ]\n logger.go:42: 11:34:19 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344848 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344855 | controller | 11:34:19 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344862 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:20 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344869 | controller | | + '[' 2 == 0 ']'\n logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344876 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344883 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344890 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344897 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.344904 | controller | 0 ]\n ]\n logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.344911 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.344918 | controller | 11:34:21 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.344940 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344948 | controller | | Error from server (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.344955 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344962 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.344970 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.344977 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.344984 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.344996 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.345004 | controller | 0 ]\n ]\n logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.345011 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.345018 | controller | 11:34:22 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.345028 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345035 | controller | | Error from server (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.345042 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345049 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:23 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345056 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.345063 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.345100 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.345108 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.345115 | controller | 0 ]\n ]\n logger.go:42: 11:34:23 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.345122 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.345129 | controller | 11:34:23 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.345136 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:24 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345143 | controller | | Error from server (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.345151 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:24 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345158 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345165 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.345172 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.345179 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.345186 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.345193 | controller | 0 ]\n ]\n logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.345200 | controller | grep -c 'Block storage service is not enabled, skipping storage collector'\n logger.go:42: 2026-01-21 11:38:24.345208 | controller | 11:34:25 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.345215 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345222 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.345230 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.345238 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.345245 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.345253 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.345261 | controller | 0 ]\n ]\n logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.345270 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.345278 | controller | 11:34:26 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.345287 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346190 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:27 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346234 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346242 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.346250 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.346257 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.346264 | controller | 0 ]\n ]\n logger.go:42: 11:34:27 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.346271 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.346279 | controller | 11:34:27 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.346293 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:28 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346305 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346311 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346317 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.346323 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.346328 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.346334 | controller | 0 ]\n ]\n logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.346340 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.346346 | controller | 11:34:29 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.346352 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346358 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346364 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346370 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.346375 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.346381 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.346387 | controller | 0 ]\n ]\n logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.346393 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.346399 | controller | 11:34:30 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.346405 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346411 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:31 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346416 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346422 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.346428 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.346436 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.346450 | controller | 0 ]\n ]\n logger.go:42: 11:34:31 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.346463 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.346471 | controller | 11:34:31 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.346479 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:32 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346486 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346494 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346501 | controller | detects that there is a cinder service and\n # does not log that storage 2026-01-21 11:38:24.346508 | controller | collector is skipped\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.346514 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.346528 | controller | 0 ]\n ]\n logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder | ++ 2026-01-21 11:38:24.346547 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.346601 | controller | 11:34:33 | watcher-cinder/2-deploy-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.346611 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346639 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder 2026-01-21 11:38:24.346648 | controller | | test step completed 2-deploy-cinder\n logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346655 | controller | | starting test step 3-remove-cinder\n logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346662 | controller | | OpenStackControlPlane:watcher-kuttl-default/openstack updated\n logger.go:42: 2026-01-21 11:38:24.346677 | controller | 11:34:33 | watcher-cinder/3-remove-cinder | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.346688 | controller | \ # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.346696 | controller | service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep 2026-01-21 11:38:24.346703 | controller | -c 'Block storage service is not enabled, skipping storage collector')\" == 2 ]\n 2026-01-21 11:38:24.346710 | controller | \ ]\n logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder | ++ oc logs 2026-01-21 11:38:24.346740 | controller | -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:33 2026-01-21 11:38:24.346749 | controller | | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is not enabled, 2026-01-21 11:38:24.346756 | controller | skipping storage collector'\n logger.go:42: 11:34:34 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346764 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346773 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346781 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.346790 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.346799 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.346807 | controller | 11:34:35 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.346816 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346825 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.346834 | controller | \ logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.346842 | controller | \ logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.346851 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.346860 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.346868 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.346877 | controller | 2 ]\n ]\n logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.346885 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.346894 | controller | 11:34:36 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.346903 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346911 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346920 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.346928 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.346942 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.346951 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.346959 | controller | 11:34:37 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.346968 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.346976 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.346985 | controller | \ logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.346993 | controller | \ logger.go:42: 11:34:38 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.347002 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.347011 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.347025 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.347034 | controller | 2 ]\n ]\n logger.go:42: 11:34:38 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.347043 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.347051 | controller | 11:34:38 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.347080 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:39 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347090 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347099 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347108 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.347117 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.347125 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.347134 | controller | 11:34:40 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.347142 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347151 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.347160 | controller | \ logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.347169 | controller | \ logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.347178 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.347188 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.347197 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.347205 | controller | 2 ]\n ]\n logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.347214 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.347222 | controller | 11:34:41 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.347231 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347239 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:42 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347248 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347256 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.347274 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.347283 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.347292 | controller | 11:34:43 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.347300 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:43 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347309 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.347317 | controller | \ logger.go:42: 11:34:43 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.347326 | controller | \ logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.347334 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.347343 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.347351 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.347360 | controller | 2 ]\n ]\n logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.347368 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.347381 | controller | 11:34:44 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.347390 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347398 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347407 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347415 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.347423 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.347431 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.347440 | controller | 11:34:45 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.347448 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347457 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.347466 | controller | \ logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.347473 | controller | \ logger.go:42: 11:34:46 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.347481 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.347505 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.347514 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.347522 | controller | 2 ]\n ]\n logger.go:42: 11:34:46 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.347529 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.347537 | controller | 11:34:46 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.347545 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:47 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347553 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347560 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347568 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.347582 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.347589 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.347597 | controller | 11:34:48 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.347604 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347612 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.347619 | controller | \ logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.347627 | controller | (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.347634 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347642 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347650 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347657 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.347665 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.347673 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.347680 | controller | 11:34:49 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.347687 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347695 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.347702 | controller | \ logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.347738 | controller | (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.347872 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347892 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347902 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347910 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.347918 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.347926 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.347934 | controller | 11:34:50 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.347942 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347950 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.347959 | controller | \ logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.347967 | controller | (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.347975 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347982 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.347990 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.347997 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348005 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348017 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348026 | controller | 11:34:52 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348051 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348062 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348070 | controller | \ logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.348078 | controller | (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.348086 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348095 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348103 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.348112 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348120 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348128 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348135 | controller | 11:34:53 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348144 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348152 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348160 | controller | \ logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.348168 | controller | (BadRequest): container \"watcher-decision-engine\" in pod \"watcher-kuttl-decision-engine-0\" 2026-01-21 11:38:24.348176 | controller | is waiting to start: ContainerCreating\n logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348184 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348192 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.348199 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348207 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348215 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348222 | controller | 11:34:54 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348231 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348241 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348255 | controller | \ logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.348265 | controller | \ logger.go:42: 11:34:55 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.348273 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.348283 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.348292 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.348299 | controller | 2 ]\n ]\n logger.go:42: 11:34:55 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.348307 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.348315 | controller | 11:34:55 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.348324 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:34:56 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348337 | controller | | + '[' 0 == 2 ']'\n logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348345 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.348352 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348359 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348367 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348375 | controller | 11:34:57 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348384 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348391 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348399 | controller | \ logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder | + '[' 0 == 2 ']'\n 2026-01-21 11:38:24.348406 | controller | \ logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.348414 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.348427 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.348452 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.348463 | controller | 2 ]\n ]\n logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.348471 | controller | grep -c 'Block storage service is not enabled, skipping storage collector'\n logger.go:42: 2026-01-21 11:38:24.348480 | controller | 11:34:58 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348488 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348496 | controller | | + '[' 1 == 2 ']'\n logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348504 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.348511 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348519 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348526 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348534 | controller | 11:34:59 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348542 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348550 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348558 | controller | \ logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder | + '[' 1 == 2 ']'\n 2026-01-21 11:38:24.348567 | controller | \ logger.go:42: 11:35:00 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.348575 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.348583 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.348591 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.348604 | controller | 2 ]\n ]\n logger.go:42: 11:35:00 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.348612 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.348620 | controller | 11:35:00 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.348628 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:35:01 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348640 | controller | | + '[' 1 == 2 ']'\n logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348649 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.348656 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348665 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348672 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348680 | controller | 11:35:02 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348689 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348698 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348705 | controller | \ logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder | + '[' 1 == 2 ']'\n 2026-01-21 11:38:24.348733 | controller | \ logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder | running command: [sh 2026-01-21 11:38:24.348748 | controller | -c set -euxo pipefail\n # check that the decision engine correctly detects 2026-01-21 11:38:24.348756 | controller | that there is no cinder service\n [ \"$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.348764 | controller | |grep -c 'Block storage service is not enabled, skipping storage collector')\" == 2026-01-21 11:38:24.348771 | controller | 2 ]\n ]\n logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder | ++ 2026-01-21 11:38:24.348779 | controller | oc logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0\n logger.go:42: 2026-01-21 11:38:24.348786 | controller | 11:35:03 | watcher-cinder/3-remove-cinder | ++ grep -c 'Block storage service is 2026-01-21 11:38:24.348794 | controller | not enabled, skipping storage collector'\n logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348801 | controller | | + '[' 1 == 2 ']'\n logger.go:42: 11:35:04 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348808 | controller | | running command: [sh -c set -euxo pipefail\n # check that the decision 2026-01-21 11:38:24.348816 | controller | engine correctly detects that there is no cinder service\n [ \"$(oc logs 2026-01-21 11:38:24.348872 | controller | -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c 'Block storage service is 2026-01-21 11:38:24.348883 | controller | not enabled, skipping storage collector')\" == 2 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.348892 | controller | 11:35:04 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.348900 | controller | watcher-kuttl-decision-engine-0\n logger.go:42: 11:35:04 | watcher-cinder/3-remove-cinder 2026-01-21 11:38:24.348924 | controller | | ++ grep -c 'Block storage service is not enabled, skipping storage collector'\n 2026-01-21 11:38:24.348933 | controller | \ logger.go:42: 11:35:05 | watcher-cinder/3-remove-cinder | + '[' 2 == 2 ']'\n 2026-01-21 11:38:24.348941 | controller | \ logger.go:42: 11:35:05 | watcher-cinder/3-remove-cinder | test step completed 2026-01-21 11:38:24.348949 | controller | 3-remove-cinder\n logger.go:42: 11:35:05 | watcher-cinder/4-cleanup-watcher | 2026-01-21 11:38:24.348957 | controller | starting test step 4-cleanup-watcher\n logger.go:42: 11:35:16 | watcher-cinder/4-cleanup-watcher 2026-01-21 11:38:24.348964 | controller | | test step completed 4-cleanup-watcher\n logger.go:42: 11:35:16 | watcher-cinder 2026-01-21 11:38:24.348972 | controller | | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher-api-scaling\n 2026-01-21 11:38:24.348979 | controller | \ logger.go:42: 11:35:16 | watcher-api-scaling | Skipping creation of user-supplied 2026-01-21 11:38:24.348987 | controller | namespace: watcher-kuttl-default\n logger.go:42: 11:35:16 | watcher-api-scaling/0-cleanup-watcher 2026-01-21 11:38:24.348995 | controller | | starting test step 0-cleanup-watcher\n logger.go:42: 11:35:16 | watcher-api-scaling/0-cleanup-watcher 2026-01-21 11:38:24.349009 | controller | | test step completed 0-cleanup-watcher\n logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349017 | controller | | starting test step 1-deploy-with-defaults\n logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349025 | controller | | Watcher:watcher-kuttl-default/watcher-kuttl created\n logger.go:42: 11:35:16 2026-01-21 11:38:24.349038 | controller | | watcher-api-scaling/1-deploy-with-defaults | running command: [sh -c set -euxo 2026-01-21 11:38:24.349047 | controller | pipefail\n oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.349056 | controller | service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc 2026-01-21 11:38:24.349064 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.349072 | controller | -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n [ $(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.349081 | controller | keystoneservice watcher -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ 2026-01-21 11:38:24.349089 | controller | -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" 2026-01-21 11:38:24.349098 | controller | ]\n # If we are running the container locally, skip following test\n if 2026-01-21 11:38:24.349106 | controller | [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.349114 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.349122 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.349130 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.349139 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.349147 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.349155 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.349163 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.349172 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349182 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.349190 | controller | value -c Name -c Type\n logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349199 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:35:21 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349206 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:35:23 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349214 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.349222 | controller | openstackclient -- openstack service list -f value -c Name -c Type |[ $(grep -c 2026-01-21 11:38:24.349230 | controller | ^watcher) == 1 ]\n SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.349238 | controller | -- openstack service list -f value -c Name -c Type -c ID | grep watcher| awk '{print 2026-01-21 11:38:24.349246 | controller | $1}')\n [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.349254 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.349262 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.349269 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.349291 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.349306 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.349314 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.349323 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.349332 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.349340 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.349349 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.349358 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.349366 | controller | \ ]\n logger.go:42: 11:35:23 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349375 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.349383 | controller | value -c Name -c Type\n logger.go:42: 11:35:23 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349391 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:35:26 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349400 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:35:27 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349408 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.349416 | controller | openstackclient -- openstack service list -f value -c Name -c Type |[ $(grep -c 2026-01-21 11:38:24.349424 | controller | ^watcher) == 1 ]\n SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.349432 | controller | -- openstack service list -f value -c Name -c Type -c ID | grep watcher| awk '{print 2026-01-21 11:38:24.349439 | controller | $1}')\n [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.349447 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.349455 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.349463 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.349476 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.349484 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.349492 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.349500 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.349511 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.349519 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.349527 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.349535 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.349542 | controller | \ ]\n logger.go:42: 11:35:27 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349550 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.349558 | controller | value -c Name -c Type\n logger.go:42: 11:35:27 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349566 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349573 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349593 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.349602 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349610 | controller | | ++ grep watcher\n logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349618 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349626 | controller | | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c\n logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349635 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.349644 | controller | \ logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.349652 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c ']'\n logger.go:42: 2026-01-21 11:38:24.349660 | controller | 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.349687 | controller | watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n logger.go:42: 11:35:33 2026-01-21 11:38:24.349697 | controller | | watcher-api-scaling/1-deploy-with-defaults | + '[' -n '' ']'\n logger.go:42: 2026-01-21 11:38:24.349705 | controller | 11:35:35 | watcher-api-scaling/1-deploy-with-defaults | running command: [sh -c 2026-01-21 11:38:24.349735 | controller | set -euxo pipefail\n oc exec -n watcher-kuttl-default openstackclient -- 2026-01-21 11:38:24.349752 | controller | openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.349761 | controller | \ SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.349770 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n 2026-01-21 11:38:24.349777 | controller | \ [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.349785 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.349793 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.349801 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.349810 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.349818 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.349827 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.349836 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.349844 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.349852 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.349859 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.349867 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.349876 | controller | \ ]\n logger.go:42: 11:35:35 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349884 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.349892 | controller | value -c Name -c Type\n logger.go:42: 11:35:35 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349901 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349910 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349926 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.349935 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349943 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349951 | controller | | ++ grep watcher\n logger.go:42: 11:35:41 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.349980 | controller | | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c\n logger.go:42: 11:35:41 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350033 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.350046 | controller | \ logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.350055 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c ']'\n logger.go:42: 2026-01-21 11:38:24.350062 | controller | 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.350071 | controller | watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n logger.go:42: 11:35:42 2026-01-21 11:38:24.350079 | controller | | watcher-api-scaling/1-deploy-with-defaults | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.350088 | controller | ']'\n logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | ++ 2026-01-21 11:38:24.350102 | controller | oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n 2026-01-21 11:38:24.350110 | controller | \ logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.350118 | controller | '' == '' ']'\n logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350126 | controller | | + exit 0\n logger.go:42: 11:35:43 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350134 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.350141 | controller | openstackclient -- openstack service list -f value -c Name -c Type |[ $(grep -c 2026-01-21 11:38:24.350149 | controller | ^watcher) == 1 ]\n SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.350172 | controller | -- openstack service list -f value -c Name -c Type -c ID | grep watcher| awk '{print 2026-01-21 11:38:24.350182 | controller | $1}')\n [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.350191 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.350200 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.350208 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.350215 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.350223 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.350233 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.350241 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.350249 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.350256 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.350265 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.350273 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.350288 | controller | \ ]\n logger.go:42: 11:35:43 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350296 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.350303 | controller | value -c Name -c Type\n logger.go:42: 11:35:43 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350315 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350323 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350330 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350338 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.350345 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350352 | controller | | ++ grep watcher\n logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350359 | controller | | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c\n logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350367 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.350374 | controller | \ logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.350381 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c ']'\n logger.go:42: 2026-01-21 11:38:24.350389 | controller | 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.350397 | controller | watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n logger.go:42: 11:35:50 2026-01-21 11:38:24.350404 | controller | | watcher-api-scaling/1-deploy-with-defaults | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.350412 | controller | ']'\n logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | ++ 2026-01-21 11:38:24.350420 | controller | oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n 2026-01-21 11:38:24.350427 | controller | \ logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.350435 | controller | '' == '' ']'\n logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350442 | controller | | + exit 0\n logger.go:42: 11:35:52 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350450 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.350459 | controller | openstackclient -- openstack service list -f value -c Name -c Type |[ $(grep -c 2026-01-21 11:38:24.350467 | controller | ^watcher) == 1 ]\n SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.350475 | controller | -- openstack service list -f value -c Name -c Type -c ID | grep watcher| awk '{print 2026-01-21 11:38:24.350484 | controller | $1}')\n [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.350492 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.350500 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n # If we are running the container 2026-01-21 11:38:24.350512 | controller | locally, skip following test\n if [ \"$(oc get pods -n openstack-operators 2026-01-21 11:38:24.350536 | controller | -o name -l openstack.org/operator-name=watcher)\" == \"\" ]; then\n exit 2026-01-21 11:38:24.350547 | controller | 0\n fi\n env_variables=$(oc set env $(oc get pods -n openstack-operators 2026-01-21 11:38:24.350555 | controller | -o name -l openstack.org/operator-name=watcher) -n openstack-operators --list)\n 2026-01-21 11:38:24.350583 | controller | \ counter=0\n for i in ${env_variables}; do\n if echo ${i} 2026-01-21 11:38:24.350599 | controller | | grep '_URL_DEFAULT' &> /dev/null; then\n echo ${i}\n counter=$((counter 2026-01-21 11:38:24.350608 | controller | + 1))\n fi\n done\n if [ ${counter} -lt 3 ]; then\n echo 2026-01-21 11:38:24.350615 | controller | \"Error: Less than 3 _URL_DEFAULT variables found.\"\n exit 1\n else\n 2026-01-21 11:38:24.350622 | controller | \ echo \"Success: ${counter} _URL_DEFAULT variables found.\"\n fi\n 2026-01-21 11:38:24.350630 | controller | \ ]\n logger.go:42: 11:35:52 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350638 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:35:52 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350646 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.350655 | controller | value -c Name -c Type\n logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350664 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350673 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.350683 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350692 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350701 | controller | | ++ grep watcher\n logger.go:42: 11:35:58 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350710 | controller | | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c\n logger.go:42: 11:35:58 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350753 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.350762 | controller | \ logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.350771 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c ']'\n logger.go:42: 2026-01-21 11:38:24.350779 | controller | 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.350787 | controller | watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n logger.go:42: 11:35:59 2026-01-21 11:38:24.350795 | controller | | watcher-api-scaling/1-deploy-with-defaults | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.350804 | controller | ']'\n logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | ++ 2026-01-21 11:38:24.350812 | controller | oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher\n 2026-01-21 11:38:24.350820 | controller | \ logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | + '[' 2026-01-21 11:38:24.350828 | controller | '' == '' ']'\n logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350836 | controller | | + exit 0\n logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults 2026-01-21 11:38:24.350844 | controller | | test step completed 1-deploy-with-defaults\n logger.go:42: 11:35:59 | watcher-api-scaling/2-scale-up-watcher-api 2026-01-21 11:38:24.350851 | controller | | starting test step 2-scale-up-watcher-api\n logger.go:42: 11:35:59 | watcher-api-scaling/2-scale-up-watcher-api 2026-01-21 11:38:24.350859 | controller | | running command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type='json' 2026-01-21 11:38:24.350866 | controller | -p='[{\"op\": \"replace\", \"path\": \"/spec/apiServiceTemplate/replicas\", \"value\":3}]'\n 2026-01-21 11:38:24.350874 | controller | \ ]\n logger.go:42: 11:35:59 | watcher-api-scaling/2-scale-up-watcher-api 2026-01-21 11:38:24.350881 | controller | | watcher.watcher.openstack.org/watcher-kuttl patched\n logger.go:42: 11:36:12 2026-01-21 11:38:24.350889 | controller | | watcher-api-scaling/2-scale-up-watcher-api | test step completed 2-scale-up-watcher-api\n 2026-01-21 11:38:24.350902 | controller | \ logger.go:42: 11:36:12 | watcher-api-scaling/3-scale-down-watcher-api | starting 2026-01-21 11:38:24.350910 | controller | test step 3-scale-down-watcher-api\n logger.go:42: 11:36:12 | watcher-api-scaling/3-scale-down-watcher-api 2026-01-21 11:38:24.350917 | controller | | running command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type='json' 2026-01-21 11:38:24.350925 | controller | -p='[{\"op\": \"replace\", \"path\": \"/spec/apiServiceTemplate/replicas\", \"value\":1}]'\n 2026-01-21 11:38:24.350932 | controller | \ ]\n logger.go:42: 11:36:12 | watcher-api-scaling/3-scale-down-watcher-api 2026-01-21 11:38:24.350940 | controller | | watcher.watcher.openstack.org/watcher-kuttl patched\n logger.go:42: 11:36:16 2026-01-21 11:38:24.350963 | controller | | watcher-api-scaling/3-scale-down-watcher-api | test step completed 3-scale-down-watcher-api\n 2026-01-21 11:38:24.350973 | controller | \ logger.go:42: 11:36:16 | watcher-api-scaling/4-scale-down-zero-watcher-api | 2026-01-21 11:38:24.350981 | controller | starting test step 4-scale-down-zero-watcher-api\n logger.go:42: 11:36:16 | watcher-api-scaling/4-scale-down-zero-watcher-api 2026-01-21 11:38:24.350990 | controller | | running command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type='json' 2026-01-21 11:38:24.350998 | controller | -p='[{\"op\": \"replace\", \"path\": \"/spec/apiServiceTemplate/replicas\", \"value\":0}]'\n 2026-01-21 11:38:24.351006 | controller | \ ]\n logger.go:42: 11:36:16 | watcher-api-scaling/4-scale-down-zero-watcher-api 2026-01-21 11:38:24.351014 | controller | | watcher.watcher.openstack.org/watcher-kuttl patched\n logger.go:42: 11:36:17 2026-01-21 11:38:24.351022 | controller | | watcher-api-scaling/4-scale-down-zero-watcher-api | test step completed 4-scale-down-zero-watcher-api\n 2026-01-21 11:38:24.351030 | controller | \ logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher | starting test 2026-01-21 11:38:24.351045 | controller | step 5-cleanup-watcher\n logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351053 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.351061 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.351069 | controller | ]\n ]\n logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351077 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.351085 | controller | value -c Name -c Type\n logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351097 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:21 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351105 | controller | | + '[' 1 == 0 ']'\n logger.go:42: 11:36:23 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351115 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.351122 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.351175 | controller | ]\n ]\n logger.go:42: 11:36:23 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351190 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.351198 | controller | value -c Name -c Type\n logger.go:42: 11:36:23 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351205 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:27 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351213 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:36:28 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351220 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.351227 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.351234 | controller | ]\n ]\n logger.go:42: 11:36:28 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351247 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.351254 | controller | value -c Name -c Type\n logger.go:42: 11:36:28 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351261 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:32 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351267 | controller | | + '[' 0 == 0 ']'\n logger.go:42: 11:36:32 | watcher-api-scaling/5-cleanup-watcher 2026-01-21 11:38:24.351273 | controller | | test step completed 5-cleanup-watcher\n logger.go:42: 11:36:32 | watcher-api-scaling 2026-01-21 11:38:24.351278 | controller | | skipping kubernetes event logging\n=== CONT kuttl/harness/watcher\n logger.go:42: 2026-01-21 11:38:24.351284 | controller | 11:36:32 | watcher | Skipping creation of user-supplied namespace: watcher-kuttl-default\n 2026-01-21 11:38:24.351290 | controller | \ logger.go:42: 11:36:32 | watcher/0-cleanup-watcher | starting test step 0-cleanup-watcher\n 2026-01-21 11:38:24.351296 | controller | \ logger.go:42: 11:36:32 | watcher/0-cleanup-watcher | test step completed 0-cleanup-watcher\n 2026-01-21 11:38:24.351302 | controller | \ logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults | starting test step 2026-01-21 11:38:24.351308 | controller | 1-deploy-with-defaults\n logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351314 | controller | | Watcher:watcher-kuttl-default/watcher-kuttl created\n logger.go:42: 11:36:32 2026-01-21 11:38:24.351320 | controller | | watcher/1-deploy-with-defaults | running command: [sh -c set -euxo pipefail\n 2026-01-21 11:38:24.351326 | controller | \ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.351331 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n SERVICEID=$(oc exec 2026-01-21 11:38:24.351337 | controller | -n watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.351355 | controller | -c Type -c ID | grep watcher| awk '{print $1}')\n [ $(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.351362 | controller | keystoneservice watcher -o jsonpath={.status.serviceID}) == ${SERVICEID} ]\n [ 2026-01-21 11:38:24.351369 | controller | -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})\" 2026-01-21 11:38:24.351375 | controller | ]\n [ \"$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.351381 | controller | -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep -c 'ssl=1')\" == 1 ]\n [ \"$(oc 2026-01-21 11:38:24.351387 | controller | get -n watcher-kuttl-default secret watcher-kuttl-api-config-data -o jsonpath='{.data.00-default\\.conf}'|base64 2026-01-21 11:38:24.351393 | controller | -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" == 2 2026-01-21 11:38:24.351399 | controller | ]\n # If we are running the container locally, skip following test\n if 2026-01-21 11:38:24.351405 | controller | [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.351411 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.351417 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.351422 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.351428 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.351435 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.351446 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.351461 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.351469 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351483 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.351490 | controller | value -c Name -c Type\n logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351497 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:36 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351505 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:36:38 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351516 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.351524 | controller | openstackclient -- openstack service list -f value -c Name -c Type |[ $(grep -c 2026-01-21 11:38:24.351531 | controller | ^watcher) == 1 ]\n SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.351537 | controller | -- openstack service list -f value -c Name -c Type -c ID | grep watcher| awk '{print 2026-01-21 11:38:24.351544 | controller | $1}')\n [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.351554 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.351570 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.351578 | controller | secret watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep 2026-01-21 11:38:24.351585 | controller | -c 'ssl=1')\" == 1 ]\n [ \"$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.351593 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.351601 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.351608 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.351616 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.351623 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.351631 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.351638 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.351645 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.351652 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.351660 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.351667 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:36:38 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351674 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.351681 | controller | value -c Name -c Type\n logger.go:42: 11:36:38 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351707 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:41 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351730 | controller | | + '[' 0 == 1 ']'\n logger.go:42: 11:36:42 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351742 | controller | | running command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.351748 | controller | openstackclient -- openstack service list -f value -c Name -c Type |[ $(grep -c 2026-01-21 11:38:24.351754 | controller | ^watcher) == 1 ]\n SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.351759 | controller | -- openstack service list -f value -c Name -c Type -c ID | grep watcher| awk '{print 2026-01-21 11:38:24.351765 | controller | $1}')\n [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.351776 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.351782 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.351788 | controller | secret watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep 2026-01-21 11:38:24.351794 | controller | -c 'ssl=1')\" == 1 ]\n [ \"$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.351800 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.351805 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.351811 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.351817 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.351823 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.351829 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.351835 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.351841 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.351846 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.351861 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.351867 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:36:42 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351873 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.351879 | controller | value -c Name -c Type\n logger.go:42: 11:36:42 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351885 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351893 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351899 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.351905 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351911 | controller | | ++ grep watcher\n logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults | 2026-01-21 11:38:24.351917 | controller | ++ awk '{print $1}'\n logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351923 | controller | | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df\n logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351929 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.351934 | controller | \ logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | + '[' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.351940 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df ']'\n logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.351946 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.351952 | controller | \ logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | + '[' -n '' ']'\n 2026-01-21 11:38:24.351958 | controller | \ logger.go:42: 11:36:49 | watcher/1-deploy-with-defaults | running command: [sh 2026-01-21 11:38:24.351964 | controller | -c set -euxo pipefail\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.351969 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.351979 | controller | \ SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.351984 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n 2026-01-21 11:38:24.351990 | controller | \ [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.351996 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.352012 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.352019 | controller | secret watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep 2026-01-21 11:38:24.352025 | controller | -c 'ssl=1')\" == 1 ]\n [ \"$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.352031 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.352037 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.352043 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.352048 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.352054 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.352060 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.352066 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.352072 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.352078 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.352083 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.352089 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:36:49 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352095 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.352101 | controller | value -c Name -c Type\n logger.go:42: 11:36:49 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352107 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352112 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352118 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.352124 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352130 | controller | | ++ grep watcher\n logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults | 2026-01-21 11:38:24.352136 | controller | ++ awk '{print $1}'\n logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352142 | controller | | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df\n logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352148 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.352153 | controller | \ logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | + '[' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.352159 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df ']'\n logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352165 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.352171 | controller | \ logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.352181 | controller | ']'\n logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ grep -c ssl=1\n 2026-01-21 11:38:24.352187 | controller | \ logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.352193 | controller | 11:36:57 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default secret 2026-01-21 11:38:24.352198 | controller | watcher-kuttl-api-config-data -o 'jsonpath={.data.my\\.cnf}'\n logger.go:42: 2026-01-21 11:38:24.352204 | controller | 11:36:57 | watcher/1-deploy-with-defaults | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.352210 | controller | 11:36:57 | watcher/1-deploy-with-defaults | ++ grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'\n 2026-01-21 11:38:24.352216 | controller | \ logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.352222 | controller | 11:36:57 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default secret 2026-01-21 11:38:24.352228 | controller | watcher-kuttl-api-config-data -o 'jsonpath={.data.00-default\\.conf}'\n logger.go:42: 2026-01-21 11:38:24.352239 | controller | 11:36:58 | watcher/1-deploy-with-defaults | + '[' 2 == 2 ']'\n logger.go:42: 2026-01-21 11:38:24.352245 | controller | 11:36:58 | watcher/1-deploy-with-defaults | ++ oc get pods -n openstack-operators 2026-01-21 11:38:24.352252 | controller | -o name -l openstack.org/operator-name=watcher\n logger.go:42: 11:36:58 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352258 | controller | | + '[' '' == '' ']'\n logger.go:42: 11:36:58 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352274 | controller | | + exit 0\n logger.go:42: 11:36:59 | watcher/1-deploy-with-defaults | running 2026-01-21 11:38:24.352283 | controller | command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.352289 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.352295 | controller | \ SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.352301 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n 2026-01-21 11:38:24.352308 | controller | \ [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.352314 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.352319 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.352325 | controller | secret watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep 2026-01-21 11:38:24.352331 | controller | -c 'ssl=1')\" == 1 ]\n [ \"$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.352338 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.352344 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.352350 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.352356 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.352362 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.352368 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.352374 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.352380 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.352386 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.352391 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.352404 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:36:59 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352410 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.352416 | controller | value -c Name -c Type\n logger.go:42: 11:36:59 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352422 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352428 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352434 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.352440 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352445 | controller | | ++ awk '{print $1}'\n logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352451 | controller | | ++ grep watcher\n logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | 2026-01-21 11:38:24.352457 | controller | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df\n logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352463 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.352469 | controller | \ logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + '[' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.352475 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df ']'\n logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352481 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.352487 | controller | \ logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.352492 | controller | ']'\n logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get -n 2026-01-21 11:38:24.352498 | controller | watcher-kuttl-default secret watcher-kuttl-api-config-data -o 'jsonpath={.data.my\\.cnf}'\n 2026-01-21 11:38:24.352504 | controller | \ logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.352510 | controller | 11:37:05 | watcher/1-deploy-with-defaults | ++ grep -c ssl=1\n logger.go:42: 2026-01-21 11:38:24.352516 | controller | 11:37:05 | watcher/1-deploy-with-defaults | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.352522 | controller | 11:37:05 | watcher/1-deploy-with-defaults | ++ base64 -d\n logger.go:42: 11:37:05 2026-01-21 11:38:24.352528 | controller | | watcher/1-deploy-with-defaults | ++ grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'\n 2026-01-21 11:38:24.352546 | controller | \ logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.352554 | controller | secret watcher-kuttl-api-config-data -o 'jsonpath={.data.00-default\\.conf}'\n logger.go:42: 2026-01-21 11:38:24.352562 | controller | 11:37:05 | watcher/1-deploy-with-defaults | + '[' 2 == 2 ']'\n logger.go:42: 2026-01-21 11:38:24.352570 | controller | 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get pods -n openstack-operators 2026-01-21 11:38:24.352578 | controller | -o name -l openstack.org/operator-name=watcher\n logger.go:42: 11:37:06 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352585 | controller | | + '[' '' == '' ']'\n logger.go:42: 11:37:06 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352593 | controller | | + exit 0\n logger.go:42: 11:37:07 | watcher/1-deploy-with-defaults | running 2026-01-21 11:38:24.352600 | controller | command: [sh -c set -euxo pipefail\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.352607 | controller | -- openstack service list -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]\n 2026-01-21 11:38:24.352618 | controller | \ SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.352629 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk '{print $1}')\n 2026-01-21 11:38:24.352636 | controller | \ [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.352643 | controller | == ${SERVICEID} ]\n [ -n \"$(oc get -n watcher-kuttl-default watcher watcher-kuttl 2026-01-21 11:38:24.352649 | controller | -o jsonpath={.status.hash.dbsync})\" ]\n [ \"$(oc get -n watcher-kuttl-default 2026-01-21 11:38:24.352655 | controller | secret watcher-kuttl-api-config-data -o jsonpath='{.data.my\\.cnf}'|base64 -d|grep 2026-01-21 11:38:24.352661 | controller | -c 'ssl=1')\" == 1 ]\n [ \"$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.352667 | controller | -o jsonpath='{.data.00-default\\.conf}'|base64 -d|grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem')\" 2026-01-21 11:38:24.352672 | controller | == 2 ]\n # If we are running the container locally, skip following test\n 2026-01-21 11:38:24.352678 | controller | \ if [ \"$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)\" 2026-01-21 11:38:24.352684 | controller | == \"\" ]; then\n exit 0\n fi\n env_variables=$(oc set 2026-01-21 11:38:24.352690 | controller | env $(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher) 2026-01-21 11:38:24.352696 | controller | -n openstack-operators --list)\n counter=0\n for i in ${env_variables}; 2026-01-21 11:38:24.352702 | controller | do\n if echo ${i} | grep '_URL_DEFAULT' &> /dev/null; then\n echo 2026-01-21 11:38:24.352708 | controller | ${i}\n counter=$((counter + 1))\n fi\n done\n if 2026-01-21 11:38:24.352728 | controller | [ ${counter} -lt 3 ]; then\n echo \"Error: Less than 3 _URL_DEFAULT variables 2026-01-21 11:38:24.352738 | controller | found.\"\n exit 1\n else\n echo \"Success: ${counter} _URL_DEFAULT 2026-01-21 11:38:24.352744 | controller | variables found.\"\n fi\n ]\n logger.go:42: 11:37:07 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352750 | controller | | + oc exec -n watcher-kuttl-default openstackclient -- openstack service list -f 2026-01-21 11:38:24.352756 | controller | value -c Name -c Type\n logger.go:42: 11:37:07 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352762 | controller | | ++ grep -c '^watcher'\n logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352767 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352773 | controller | | ++ grep watcher\n logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults | 2026-01-21 11:38:24.352779 | controller | ++ awk '{print $1}'\n logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352785 | controller | | ++ oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.352791 | controller | -f value -c Name -c Type -c ID\n logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352797 | controller | | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df\n logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352803 | controller | | ++ oc get -n watcher-kuttl-default keystoneservice watcher -o 'jsonpath={.status.serviceID}'\n 2026-01-21 11:38:24.352809 | controller | \ logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | + '[' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.352814 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df ']'\n logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352820 | controller | | ++ oc get -n watcher-kuttl-default watcher watcher-kuttl -o 'jsonpath={.status.hash.dbsync}'\n 2026-01-21 11:38:24.352826 | controller | \ logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | + '[' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.352842 | controller | ']'\n logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ oc get -n 2026-01-21 11:38:24.352853 | controller | watcher-kuttl-default secret watcher-kuttl-api-config-data -o 'jsonpath={.data.my\\.cnf}'\n 2026-01-21 11:38:24.352859 | controller | \ logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ base64 -d\n logger.go:42: 2026-01-21 11:38:24.352865 | controller | 11:37:12 | watcher/1-deploy-with-defaults | ++ grep -c ssl=1\n logger.go:42: 2026-01-21 11:38:24.352871 | controller | 11:37:12 | watcher/1-deploy-with-defaults | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.352877 | controller | 11:37:12 | watcher/1-deploy-with-defaults | ++ base64 -d\n logger.go:42: 11:37:12 2026-01-21 11:38:24.352883 | controller | | watcher/1-deploy-with-defaults | ++ grep -c 'cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'\n 2026-01-21 11:38:24.352889 | controller | \ logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.352895 | controller | secret watcher-kuttl-api-config-data -o 'jsonpath={.data.00-default\\.conf}'\n logger.go:42: 2026-01-21 11:38:24.352901 | controller | 11:37:13 | watcher/1-deploy-with-defaults | + '[' 2 == 2 ']'\n logger.go:42: 2026-01-21 11:38:24.352907 | controller | 11:37:13 | watcher/1-deploy-with-defaults | ++ oc get pods -n openstack-operators 2026-01-21 11:38:24.352914 | controller | -o name -l openstack.org/operator-name=watcher\n logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352925 | controller | | + '[' '' == '' ']'\n logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults 2026-01-21 11:38:24.352940 | controller | | + exit 0\n logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults | test step 2026-01-21 11:38:24.352948 | controller | completed 1-deploy-with-defaults\n logger.go:42: 11:37:13 | watcher/2-cleanup-watcher 2026-01-21 11:38:24.352956 | controller | | starting test step 2-cleanup-watcher\n logger.go:42: 11:37:13 | watcher/2-cleanup-watcher 2026-01-21 11:38:24.352969 | controller | | test step completed 2-cleanup-watcher\n logger.go:42: 11:37:13 | watcher/3-precreate-mariadbaccount 2026-01-21 11:38:24.352978 | controller | | starting test step 3-precreate-mariadbaccount\n logger.go:42: 11:37:13 | watcher/3-precreate-mariadbaccount 2026-01-21 11:38:24.352985 | controller | | MariaDBAccount:watcher-kuttl-default/watcher-precreated created\n logger.go:42: 2026-01-21 11:38:24.352992 | controller | 11:37:13 | watcher/3-precreate-mariadbaccount | test step completed 3-precreate-mariadbaccount\n 2026-01-21 11:38:24.353000 | controller | \ logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | starting 2026-01-21 11:38:24.353007 | controller | test step 4-deploy-with-precreated-account\n logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353014 | controller | | Secret:watcher-kuttl-default/custom-prometheus-config created\n logger.go:42: 2026-01-21 11:38:24.353021 | controller | 11:37:13 | watcher/4-deploy-with-precreated-account | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.353028 | controller | created\n logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353034 | controller | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.353043 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.353049 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.353055 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.353061 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.353066 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.353073 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.353118 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.353144 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.353151 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.353158 | controller | \ logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.353166 | controller | watcher-kuttl-default\n logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353173 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.353180 | controller | \ logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.353187 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.353193 | controller | \ logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.353214 | controller | -1\n logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.353220 | controller | -v '^$'\n logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353226 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353232 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353238 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353244 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.353250 | controller | \ logger.go:42: 11:37:15 | watcher/4-deploy-with-precreated-account | error: Internal 2026-01-21 11:38:24.353256 | controller | error occurred: error executing command in container: container is not created or 2026-01-21 11:38:24.353262 | controller | running\n logger.go:42: 11:37:15 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353268 | controller | ++ echo\n logger.go:42: 11:37:15 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353274 | controller | + '[' 0 == 1 ']'\n logger.go:42: 11:37:16 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353286 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.353292 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.353298 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.353304 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.353309 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.353316 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.353323 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.353340 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.353350 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.353358 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.353366 | controller | \ logger.go:42: 11:37:16 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.353379 | controller | watcher-kuttl-default\n logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353387 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.353395 | controller | \ logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.353402 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.353410 | controller | \ logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.353417 | controller | -v '^$'\n logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353424 | controller | ++ head -1\n logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353432 | controller | | + APIPOD=\n logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353439 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.353446 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.353463 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.353473 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.353481 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.353488 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.353496 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.353503 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.353510 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.353517 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.353540 | controller | \ logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.353549 | controller | watcher-kuttl-default\n logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353563 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.353576 | controller | \ logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.353584 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.353591 | controller | \ logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.353599 | controller | -v '^$'\n logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353606 | controller | ++ head -1\n logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353613 | controller | | + APIPOD=\n logger.go:42: 11:37:19 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353620 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.353626 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.353633 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.353640 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.353647 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.353661 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.353669 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.353682 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.353690 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.353697 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.353705 | controller | \ logger.go:42: 11:37:19 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.353735 | controller | watcher-kuttl-default\n logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353749 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.353756 | controller | \ logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.353764 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.353772 | controller | \ logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.353780 | controller | -1\n logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.353788 | controller | -v '^$'\n logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353796 | controller | + APIPOD=\n logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353803 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.353811 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.353819 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.353826 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.353834 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.353848 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.353856 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.353864 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.353872 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.353879 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.353887 | controller | \ logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.353894 | controller | watcher-kuttl-default\n logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353923 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.353933 | controller | \ logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.353941 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.353950 | controller | \ logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.353957 | controller | -1\n logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.353971 | controller | -v '^$'\n logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.353979 | controller | + APIPOD=\n logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.353987 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.353994 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.354002 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.354009 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.354017 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.354025 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.354033 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.354040 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.354049 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.354057 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.354064 | controller | \ logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.354072 | controller | watcher-kuttl-default\n logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354080 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.354088 | controller | \ logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.354095 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.354103 | controller | \ logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.354111 | controller | -1\n logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.354119 | controller | -v '^$'\n logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.354127 | controller | + APIPOD=\n logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354135 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.354142 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.354150 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.354157 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.354165 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.354173 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.354181 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.354195 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.354204 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.354214 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.354228 | controller | \ logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.354239 | controller | watcher-kuttl-default\n logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354248 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.354257 | controller | \ logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.354286 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.354297 | controller | \ logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.354305 | controller | -1\n logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.354314 | controller | -v '^$'\n logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.354329 | controller | + APIPOD=\n logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354338 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.354346 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.354354 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.354362 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.354370 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.354377 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.354385 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.354392 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.354401 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.354409 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.354417 | controller | \ logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.354430 | controller | watcher-kuttl-default\n logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354439 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.354447 | controller | \ logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.354455 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.354462 | controller | \ logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.354470 | controller | -1\n logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.354478 | controller | -v '^$'\n logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.354486 | controller | + APIPOD=\n logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354494 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.354502 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.354509 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.354516 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.354530 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.354538 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.354546 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.354555 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.354588 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.354596 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.354605 | controller | \ logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.354612 | controller | watcher-kuttl-default\n logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354620 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.354628 | controller | \ logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.354635 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.354643 | controller | \ logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.354665 | controller | -1\n logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.354674 | controller | -v '^$'\n logger.go:42: 11:37:28 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.354682 | controller | + APIPOD=\n logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354689 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.354697 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.354704 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.354732 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.354748 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.354762 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.354770 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.354778 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.354786 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.354795 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.354803 | controller | \ logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.354810 | controller | watcher-kuttl-default\n logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354818 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.354826 | controller | \ logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.354833 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.354847 | controller | \ logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.354855 | controller | -1\n logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.354863 | controller | -v '^$'\n logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.354870 | controller | + APIPOD=\n logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354877 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.354885 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.354893 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.354901 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.354910 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.354918 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.354925 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.354933 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.354941 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.354949 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.354957 | controller | \ logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.354964 | controller | watcher-kuttl-default\n logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.354972 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.354980 | controller | \ logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.354989 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.354997 | controller | \ logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.355005 | controller | -1\n logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.355014 | controller | -v '^$'\n logger.go:42: 11:37:31 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.355041 | controller | + APIPOD=\n logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355051 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.355060 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.355067 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.355074 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.355082 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.355090 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.355098 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.355107 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.355122 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.355132 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.355140 | controller | \ logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.355148 | controller | watcher-kuttl-default\n logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355156 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.355163 | controller | \ logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.355172 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.355186 | controller | \ logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.355194 | controller | -1\n logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.355203 | controller | -v '^$'\n logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.355211 | controller | + APIPOD=\n logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355219 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.355228 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.355238 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.355247 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.355256 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.355265 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.355272 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.355280 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.355287 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.355295 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.355303 | controller | \ logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.355310 | controller | watcher-kuttl-default\n logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355318 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.355325 | controller | \ logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.355333 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.355340 | controller | \ logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.355347 | controller | -1\n logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.355354 | controller | -v '^$'\n logger.go:42: 11:37:34 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.355362 | controller | + APIPOD=\n logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355369 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.355392 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.355414 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.355423 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.355431 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.355439 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.355446 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.355455 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.355463 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.355472 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.355479 | controller | \ logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.355486 | controller | watcher-kuttl-default\n logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355493 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.355501 | controller | \ logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.355508 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.355516 | controller | \ logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.355524 | controller | -1\n logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.355531 | controller | -v '^$'\n logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.355538 | controller | + APIPOD=\n logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355546 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.355560 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.355568 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.355590 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.355599 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.355607 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.355614 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.355623 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.355631 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.355639 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.355647 | controller | \ logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.355656 | controller | watcher-kuttl-default\n logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355664 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.355677 | controller | \ logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.355686 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.355694 | controller | \ logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.355702 | controller | -1\n logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.355710 | controller | -v '^$'\n logger.go:42: 11:37:37 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.355753 | controller | + APIPOD=\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355762 | controller | | running command: [sh -c set -euxo pipefail\n oc project watcher-kuttl-default\n 2026-01-21 11:38:24.355770 | controller | \ APIPOD=$(oc get pods -n watcher-kuttl-default -l \"service=watcher-api\" 2026-01-21 11:38:24.355779 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)\n if [ -n \"${APIPOD}\" 2026-01-21 11:38:24.355803 | controller | ]; then\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.355813 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.355822 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.355830 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.355839 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.355847 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.355855 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.355863 | controller | \ logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.355871 | controller | watcher-kuttl-default\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355880 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.355887 | controller | \ logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.355895 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.355904 | controller | \ logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.355912 | controller | -v '^$'\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.355919 | controller | ++ head -1\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355927 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355935 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355944 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.355958 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.355967 | controller | \ logger.go:42: 11:37:39 | watcher/4-deploy-with-precreated-account | error: unable 2026-01-21 11:38:24.355976 | controller | to upgrade connection: container not found (\"watcher-api\")\n logger.go:42: 2026-01-21 11:38:24.355984 | controller | 11:37:39 | watcher/4-deploy-with-precreated-account | ++ echo\n logger.go:42: 2026-01-21 11:38:24.355992 | controller | 11:37:39 | watcher/4-deploy-with-precreated-account | + '[' 0 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.356006 | controller | 11:37:40 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.356016 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.356025 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.356034 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.356044 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.356058 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.356068 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.356076 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.356111 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.356121 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.356129 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.356142 | controller | \ logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.356150 | controller | watcher-kuttl-default\n logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356158 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.356166 | controller | \ logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.356174 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.356183 | controller | \ logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.356209 | controller | -1\n logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.356219 | controller | -v '^$'\n logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.356228 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356236 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356244 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356252 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.356260 | controller | \ logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.356269 | controller | '#' Global config\n logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356277 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356286 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356295 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.356303 | controller | \ logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.356311 | controller | '#' Service config\n logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356319 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356328 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.356345 | controller | \ logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.356353 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.356361 | controller | \ logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.356371 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.356380 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.356389 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.356398 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.356407 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.356415 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.356423 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.356431 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.356438 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.356447 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.356456 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.356465 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.356474 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.356483 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.356491 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.356499 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.356506 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.356514 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.356523 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.356547 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.356557 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.356566 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.356575 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.356583 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.356592 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.356600 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.356614 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.356622 | controller | period = 900\n logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356630 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356639 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.356647 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.356655 | controller | \ logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.356663 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.356678 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.356686 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.356699 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.356707 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.356737 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.356753 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.356766 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.356774 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.356782 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.356789 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.356797 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.356805 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.356813 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.356820 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.356828 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.356835 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.356843 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.356851 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.356858 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.356866 | controller | 11:37:42 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.356873 | controller | 11:37:43 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.356881 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.356889 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.356897 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.356906 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.356920 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.356929 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.356937 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.356963 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.356974 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.356982 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.356990 | controller | \ logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.356998 | controller | watcher-kuttl-default\n logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357006 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.357014 | controller | \ logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.357022 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.357029 | controller | \ logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.357038 | controller | -1\n logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.357045 | controller | -v '^$'\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.357053 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357060 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357067 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357076 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.357085 | controller | \ logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.357095 | controller | '#' Global config\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357104 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357114 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357123 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.357132 | controller | \ logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.357141 | controller | '#' Service config\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357150 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357159 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.357167 | controller | \ logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.357176 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.357184 | controller | \ logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.357192 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.357206 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.357215 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.357224 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.357234 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.357243 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.357252 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.357263 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.357272 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.357281 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.357290 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.357318 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.357328 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.357336 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.357343 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.357351 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.357358 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.357365 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.357373 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.357380 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.357388 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.357395 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.357403 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.357411 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.357418 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.357425 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.357433 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.357440 | controller | period = 900\n logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357458 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357468 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357476 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.357489 | controller | \ logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.357498 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.357514 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.357521 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.357529 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.357537 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.357545 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.357553 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.357560 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.357568 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.357576 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.357589 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.357603 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.357611 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.357619 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.357627 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.357634 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.357641 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.357648 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.357656 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.357663 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.357689 | controller | 11:37:45 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.357698 | controller | 11:37:46 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.357705 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.357733 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.357747 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.357755 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.357762 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.357770 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.357777 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.357785 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.357799 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.357807 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.357815 | controller | \ logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.357823 | controller | watcher-kuttl-default\n logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357830 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.357838 | controller | \ logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.357852 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.357860 | controller | \ logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.357867 | controller | -1\n logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.357875 | controller | -v '^$'\n logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.357883 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357890 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357898 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357906 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.357913 | controller | \ logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.357921 | controller | '#' Global config\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357939 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357948 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357956 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.357965 | controller | \ logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.357973 | controller | '#' Service config\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357981 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.357989 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.357996 | controller | \ logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.358003 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.358010 | controller | \ logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.358017 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.358025 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.358032 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.358061 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.358076 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.358084 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.358092 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.358100 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.358108 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.358115 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.358123 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.358136 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.358143 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.358150 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.358157 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.358163 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.358169 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.358175 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.358181 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.358187 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.358193 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.358199 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.358205 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.358211 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.358217 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.358223 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.358228 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.358234 | controller | period = 900\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358240 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358246 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358252 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.358258 | controller | \ logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.358264 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.358276 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.358282 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.358291 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.358297 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.358303 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.358309 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.358315 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.358321 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.358327 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.358344 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.358352 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.358358 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.358364 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.358373 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.358379 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.358385 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.358391 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.358397 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.358403 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.358409 | controller | 11:37:47 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.358415 | controller | 11:37:49 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.358421 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.358427 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.358433 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.358439 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.358445 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.358451 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.358457 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.358463 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.358469 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.358475 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.358482 | controller | \ logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.358488 | controller | watcher-kuttl-default\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358494 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.358503 | controller | \ logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.358509 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.358515 | controller | \ logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.358521 | controller | -1\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.358527 | controller | -v '^$'\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.358533 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358539 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358547 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358554 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.358649 | controller | \ logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.358670 | controller | '#' Global config\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358679 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358686 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358693 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.358701 | controller | \ logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.358709 | controller | '#' Service config\n logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358761 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.358798 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.358807 | controller | \ logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.358813 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.358819 | controller | \ logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.358825 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.358831 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.358837 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.358843 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.358849 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.358856 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.358863 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.358871 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.358878 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.358891 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.358899 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.358906 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.358913 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.358921 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.358928 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.358936 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.358943 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.358950 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.358958 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.358965 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.358972 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.358980 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.358987 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.358999 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.359007 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.359014 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.359022 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.359029 | controller | period = 900\n logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359036 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359044 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359052 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.359059 | controller | \ logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.359067 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.359100 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.359108 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.359115 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.359122 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.359129 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.359137 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.359145 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.359158 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.359166 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.359174 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.359184 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.359192 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.359199 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.359207 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.359214 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.359221 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.359228 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.359235 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.359242 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.359249 | controller | 11:37:50 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.359257 | controller | 11:37:51 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.359264 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.359272 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.359279 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.359287 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.359295 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.359312 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.359320 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.359328 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.359336 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.359343 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.359350 | controller | \ logger.go:42: 11:37:51 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.359358 | controller | watcher-kuttl-default\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359365 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.359372 | controller | \ logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.359379 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.359386 | controller | \ logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.359394 | controller | -1\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.359402 | controller | -v '^$'\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.359415 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359423 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359431 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359460 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.359470 | controller | \ logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.359487 | controller | '#' Global config\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359498 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359506 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359513 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.359521 | controller | \ logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.359529 | controller | '#' Service config\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359536 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359544 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.359551 | controller | \ logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.359557 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.359562 | controller | \ logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.359568 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.359574 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.359580 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.359586 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.359593 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.359600 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.359608 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.359615 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.359623 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.359630 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.359638 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.359646 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.359653 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.359666 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.359675 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.359683 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.359690 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.359698 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.359706 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.359733 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.359746 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.359752 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.359757 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.359763 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.359769 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.359788 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.359794 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.359800 | controller | period = 900\n logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359806 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359812 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.359818 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.359824 | controller | \ logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.359831 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.359846 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.359853 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.359861 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.359868 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.359876 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.359883 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.359891 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.359899 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.359907 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.359915 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.359922 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.359930 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.359943 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.359951 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.359958 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.359966 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.359973 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.359981 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.359988 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.359996 | controller | 11:37:53 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.360003 | controller | 11:37:54 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.360015 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.360023 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.360031 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.360037 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.360043 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.360049 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.360055 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.360061 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.360067 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.360076 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.360082 | controller | \ logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.360100 | controller | watcher-kuttl-default\n logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360106 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.360112 | controller | \ logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.360118 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.360124 | controller | \ logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.360130 | controller | -1\n logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.360136 | controller | -v '^$'\n logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.360142 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360148 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360154 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360160 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.360169 | controller | \ logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360175 | controller | '#' Global config\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360181 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360187 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360193 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.360199 | controller | \ logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360205 | controller | '#' Service config\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360211 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360217 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.360223 | controller | \ logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.360230 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.360237 | controller | \ logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360246 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.360252 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.360259 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.360267 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.360273 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.360280 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.360287 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.360293 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.360299 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.360305 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.360311 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.360320 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.360325 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.360331 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.360337 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.360343 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.360361 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.360370 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.360380 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.360385 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.360391 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.360398 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.360403 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.360410 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.360417 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.360424 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.360431 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.360438 | controller | period = 900\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360445 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360452 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360460 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.360467 | controller | \ logger.go:42: 11:37:56 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360475 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.360489 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.360497 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.360504 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.360511 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.360518 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.360526 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.360533 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.360541 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.360548 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.360556 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.360563 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.360571 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.360584 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.360593 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.360600 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.360607 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.360615 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.360627 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.360634 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.360643 | controller | 11:37:56 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.360651 | controller | 11:37:57 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.360658 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.360665 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.360673 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.360681 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.360702 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.360735 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.360748 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.360755 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.360764 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.360771 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.360779 | controller | \ logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.360786 | controller | watcher-kuttl-default\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360794 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.360801 | controller | \ logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.360809 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.360816 | controller | \ logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.360824 | controller | -v '^$'\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.360832 | controller | ++ head -1\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360839 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360847 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360855 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360862 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.360870 | controller | \ logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360878 | controller | '#' Global config\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360885 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360893 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360900 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.360912 | controller | \ logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360920 | controller | '#' Service config\n logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360927 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.360935 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.360942 | controller | \ logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.360950 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.360958 | controller | \ logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.360966 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.360980 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.360988 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.360999 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.361007 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.361014 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.361022 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.361042 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.361052 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.361059 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.361066 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.361073 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.361082 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.361089 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.361097 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.361103 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.361109 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.361115 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.361121 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.361127 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.361133 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.361138 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.361144 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.361156 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.361162 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.361168 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.361174 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.361180 | controller | period = 900\n logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361188 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361194 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361199 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.361205 | controller | \ logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.361215 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.361231 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.361238 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.361246 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.361253 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.361259 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.361265 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.361271 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.361276 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.361282 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.361289 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.361296 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.361308 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.361322 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.361329 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.361336 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.361356 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.361365 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.361372 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.361379 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.361562 | controller | 11:37:58 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.361580 | controller | 11:37:59 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.361588 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.361608 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.361622 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.361628 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.361634 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.361640 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.361651 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.361657 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.361663 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.361669 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.361675 | controller | \ logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.361680 | controller | watcher-kuttl-default\n logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361686 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.361692 | controller | \ logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.361698 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.361704 | controller | \ logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.361709 | controller | -v '^$'\n logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.361738 | controller | ++ head -1\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361745 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361751 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361757 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361763 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.361769 | controller | \ logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.361774 | controller | '#' Global config\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361780 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361786 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361792 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.361798 | controller | \ logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.361803 | controller | '#' Service config\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361809 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.361819 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.361828 | controller | \ logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.361834 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.361840 | controller | \ logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.361860 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.361866 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.361872 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.361878 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.361884 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.361890 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.361896 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.361902 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.361908 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.361913 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.361919 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.361925 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.361931 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.361937 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.361943 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.361951 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.361957 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.361963 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.361969 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.361975 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.361981 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.361986 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.361992 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.361998 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.362004 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.362010 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.362015 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.362021 | controller | period = 900\n logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362030 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362036 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362042 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.362048 | controller | \ logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.362053 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.362065 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.362071 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.362077 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.362082 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.362088 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.362097 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.362111 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.362117 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.362123 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.362129 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.362135 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.362141 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.362146 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.362152 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.362158 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.362164 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.362170 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.362175 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.362181 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.362187 | controller | 11:38:01 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.362193 | controller | 11:38:02 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.362199 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.362205 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.362210 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.362216 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.362222 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.362228 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.362240 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.362246 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.362252 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.362258 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.362266 | controller | \ logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.362273 | controller | watcher-kuttl-default\n logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362279 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.362286 | controller | \ logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.362293 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.362299 | controller | \ logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.362304 | controller | -1\n logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.362310 | controller | -v '^$'\n logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.362316 | controller | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362322 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362328 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362334 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.362340 | controller | \ logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.362346 | controller | '#' Global config\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362352 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362358 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362373 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.362379 | controller | \ logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.362385 | controller | '#' Service config\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362391 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.362397 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.362403 | controller | \ logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.362409 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.362415 | controller | \ logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.362421 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.362427 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.362447 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.362461 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.362469 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.362482 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.362490 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.362498 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.362508 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.362514 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.362520 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.362526 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.362532 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.362538 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.362545 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.362555 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.366275 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.366312 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.366320 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.366328 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.366356 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.366364 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.366372 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.366379 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.366387 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.366394 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.366401 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.366409 | controller | period = 900\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366416 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366424 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366473 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.366482 | controller | \ logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.366499 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.366514 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.366522 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.366530 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.366537 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.366544 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.366552 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.366560 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.366623 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.366632 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.366639 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.366646 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.366654 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.366661 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.366668 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.366676 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.366683 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.366690 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.366698 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.366705 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.366737 | controller | 11:38:03 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.366750 | controller | 11:38:05 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.366757 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.366765 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.366772 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.366780 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.366787 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.366794 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.366801 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.366809 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.366818 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.366825 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.366837 | controller | \ logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.366845 | controller | watcher-kuttl-default\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366852 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.366860 | controller | \ logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.366867 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.366875 | controller | \ logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.366887 | controller | -v '^$'\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.366908 | controller | ++ head -1\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366915 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366923 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366930 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366938 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.366945 | controller | \ logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.366953 | controller | '#' Global config\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366961 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366968 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.366980 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.367047 | controller | \ logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.367059 | controller | '#' Service config\n logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367066 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367074 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.367082 | controller | \ logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.367095 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.367103 | controller | \ logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.367111 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.367119 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.367126 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.367134 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.367142 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.367150 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.367162 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.367170 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.367177 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.367184 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.367192 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.367199 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.367207 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.367214 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.367222 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.367229 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.367236 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.367246 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.367253 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.367261 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.367283 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.367293 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.367302 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.367310 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.367318 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.367325 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.367333 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.367340 | controller | period = 900\n logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367348 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367355 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367363 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.367370 | controller | \ logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.367377 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.367392 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.367400 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.367407 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.367414 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.367426 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.367433 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.367441 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.367448 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.367456 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.367463 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.367471 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.367478 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.367484 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.367491 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.367497 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.367667 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.367675 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.367683 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.367691 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.367698 | controller | 11:38:06 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.367706 | controller | 11:38:07 | watcher/4-deploy-with-precreated-account | running command: [sh -c set 2026-01-21 11:38:24.367731 | controller | -euxo pipefail\n oc project watcher-kuttl-default\n APIPOD=$(oc get 2026-01-21 11:38:24.367742 | controller | pods -n watcher-kuttl-default -l \"service=watcher-api\" -ocustom-columns=:metadata.name|grep 2026-01-21 11:38:24.367750 | controller | -v ^$|head -1)\n if [ -n \"${APIPOD}\" ]; then\n [ $(echo $(oc rsh 2026-01-21 11:38:24.367757 | controller | -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.367765 | controller | |grep -c \"^# Global config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api 2026-01-21 11:38:24.367772 | controller | ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) |grep -c \"^# 2026-01-21 11:38:24.367779 | controller | Service config\") == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat 2026-01-21 11:38:24.367799 | controller | /etc/watcher/watcher.conf.d/00-default.conf) |grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem') 2026-01-21 11:38:24.367807 | controller | == 1 ]\n [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.367814 | controller | |grep -czPo 'TimeOut 80') == 1 ]\n else\n exit 1\n fi\n ]\n 2026-01-21 11:38:24.367822 | controller | \ logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.367830 | controller | watcher-kuttl-default\n logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367837 | controller | | Already on project \"watcher-kuttl-default\" on server \"https://api.crc.testing:6443\".\n 2026-01-21 11:38:24.367845 | controller | \ logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.367852 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name\n 2026-01-21 11:38:24.367864 | controller | \ logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.367909 | controller | -v '^$'\n logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | 2026-01-21 11:38:24.367916 | controller | ++ head -1\n logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367927 | controller | | + APIPOD=watcher-kuttl-api-0\n logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367935 | controller | | + '[' -n watcher-kuttl-api-0 ']'\n logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367942 | controller | | ++ grep -c '^# Global config'\n logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367950 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf\n 2026-01-21 11:38:24.367957 | controller | \ logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.367965 | controller | '#' Global config\n logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367972 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367979 | controller | | ++ grep -c '^# Service config'\n logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.367987 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf\n 2026-01-21 11:38:24.367994 | controller | \ logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.368002 | controller | '#' Service config\n logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.368009 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.368016 | controller | | ++ grep -czPo '\\[prometheus_client\\]\\s+host\\s+=\\s+metric-storage-prometheus.watcher-kuttl-default.svc\\s+port\\s+=\\s+9090\\s+cafile\\s+=\\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'\n 2026-01-21 11:38:24.368024 | controller | \ logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | +++ oc rsh 2026-01-21 11:38:24.368031 | controller | -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf\n 2026-01-21 11:38:24.368039 | controller | \ logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.368046 | controller | '[DEFAULT]' state_path = /var/lib/watcher transport_url = 'rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1' 2026-01-21 11:38:24.368053 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.368061 | controller | '#' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.368069 | controller | = '[database]' connection = 'mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf' 2026-01-21 11:38:24.368076 | controller | '[oslo_policy]' policy_file = /etc/watcher/policy.yaml.sample '[oslo_messaging_notifications]' 2026-01-21 11:38:24.368084 | controller | driver = noop '[oslo_messaging_rabbit]' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.368091 | controller | amqp_durable_queues=true '[keystone_authtoken]' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.368099 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.368106 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.368114 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.368121 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.368162 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.368188 | controller | '[watcher_clients_auth]' project_domain_name = Default project_name = service user_domain_name 2026-01-21 11:38:24.368197 | controller | = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.368206 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.368213 | controller | '[oslo_concurrency]' lock_path = /var/lib/watcher/tmp '[watcher_datasources]' datasources 2026-01-21 11:38:24.368221 | controller | = prometheus '[cache]' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.368239 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.368246 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.368253 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 '[prometheus_client]' 2026-01-21 11:38:24.368261 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.368268 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem '[cinder_client]' 2026-01-21 11:38:24.368275 | controller | endpoint_type = internal '[glance_client]' endpoint_type = internal '[ironic_client]' 2026-01-21 11:38:24.368282 | controller | endpoint_type = internal '[keystone_client]' interface = internal '[neutron_client]' 2026-01-21 11:38:24.368290 | controller | endpoint_type = internal '[nova_client]' endpoint_type = internal '[placement_client]' 2026-01-21 11:38:24.368302 | controller | interface = internal '[watcher_cluster_data_model_collectors.compute]' period = 2026-01-21 11:38:24.368309 | controller | 900 '[watcher_cluster_data_model_collectors.baremetal]' period = 900 '[watcher_cluster_data_model_collectors.storage]' 2026-01-21 11:38:24.368317 | controller | period = 900\n logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.368324 | controller | | + '[' 1 == 1 ']'\n logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.368332 | controller | | ++ grep -czPo 'TimeOut 80'\n logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account 2026-01-21 11:38:24.368339 | controller | | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf\n 2026-01-21 11:38:24.368347 | controller | \ logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.368354 | controller | '#' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '' ServerName watcher-internal.watcher-kuttl-default.svc '##' Vhost docroot 2026-01-21 11:38:24.368550 | controller | DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.368558 | controller | 80 '##' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.368565 | controller | '' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.368573 | controller | AllowOverride None Require all granted '' '##' Logging ErrorLog /dev/stdout 2026-01-21 11:38:24.368580 | controller | ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' CustomLog /dev/stdout 2026-01-21 11:38:24.368587 | controller | proxy env=forwarded '##' set watcher log level to debug LogLevel debug '##' WSGI 2026-01-21 11:38:24.368594 | controller | configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess internal display-name=internal 2026-01-21 11:38:24.368602 | controller | group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup internal WSGIScriptAlias 2026-01-21 11:38:24.368609 | controller | / '\"/usr/bin/watcher-api-wsgi\"' '' '#' public vhost watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.368622 | controller | configuration '' ServerName watcher-public.watcher-kuttl-default.svc 2026-01-21 11:38:24.368629 | controller | '##' Vhost docroot DocumentRoot '\"/var/www/cgi-bin\"' '#' Set the timeout for the 2026-01-21 11:38:24.368637 | controller | watcher-api TimeOut 80 '##' Directories, there should at least be a declaration 2026-01-21 11:38:24.368644 | controller | for /var/www/cgi-bin '' Options -Indexes +FollowSymLinks 2026-01-21 11:38:24.368651 | controller | +MultiViews AllowOverride None Require all granted '' '##' Logging ErrorLog 2026-01-21 11:38:24.368658 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined 'env=!forwarded' 2026-01-21 11:38:24.368665 | controller | CustomLog /dev/stdout proxy env=forwarded '##' set watcher log level to debug LogLevel 2026-01-21 11:38:24.368673 | controller | debug '##' WSGI configuration WSGIApplicationGroup '%{GLOBAL}' WSGIDaemonProcess 2026-01-21 11:38:24.368680 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.368688 | controller | public WSGIScriptAlias / '\"/usr/bin/watcher-api-wsgi\"' ''\n logger.go:42: 2026-01-21 11:38:24.368699 | controller | 11:38:10 | watcher/4-deploy-with-precreated-account | + '[' 1 == 1 ']'\n logger.go:42: 2026-01-21 11:38:24.368739 | controller | 11:38:10 | watcher/4-deploy-with-precreated-account | test step completed 4-deploy-with-precreated-account\n 2026-01-21 11:38:24.368753 | controller | \ logger.go:42: 11:38:10 | watcher/5-cleanup-watcher | starting test step 5-cleanup-watcher\n 2026-01-21 11:38:24.368761 | controller | \ logger.go:42: 11:38:10 | watcher/5-cleanup-watcher | running command: [sh -c 2026-01-21 11:38:24.368769 | controller | set -ex\n oc exec -n watcher-kuttl-default openstackclient -- openstack service 2026-01-21 11:38:24.368776 | controller | list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]\n ]\n logger.go:42: 2026-01-21 11:38:24.368783 | controller | 11:38:10 | watcher/5-cleanup-watcher | + oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.368791 | controller | -- openstack service list -f value -c Name -c Type\n logger.go:42: 11:38:10 | 2026-01-21 11:38:24.368798 | controller | watcher/5-cleanup-watcher | ++ grep -c '^watcher'\n logger.go:42: 11:38:15 | 2026-01-21 11:38:24.368805 | controller | watcher/5-cleanup-watcher | + '[' 1 == 0 ']'\n logger.go:42: 11:38:16 | watcher/5-cleanup-watcher 2026-01-21 11:38:24.368813 | controller | | running command: [sh -c set -ex\n oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.368820 | controller | -- openstack service list -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 2026-01-21 11:38:24.368827 | controller | ]\n ]\n logger.go:42: 11:38:16 | watcher/5-cleanup-watcher | + oc exec 2026-01-21 11:38:24.368835 | controller | -n watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.368842 | controller | -c Type\n logger.go:42: 11:38:16 | watcher/5-cleanup-watcher | ++ grep -c '^watcher'\n 2026-01-21 11:38:24.368849 | controller | \ logger.go:42: 11:38:19 | watcher/5-cleanup-watcher | + '[' 0 == 0 ']'\n logger.go:42: 2026-01-21 11:38:24.368856 | controller | 11:38:20 | watcher/5-cleanup-watcher | running command: [sh -c set -ex\n oc 2026-01-21 11:38:24.368864 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.368871 | controller | -c Name -c Type | [ $(grep -c ^watcher) == 0 ]\n ]\n logger.go:42: 11:38:20 2026-01-21 11:38:24.368879 | controller | | watcher/5-cleanup-watcher | + oc exec -n watcher-kuttl-default openstackclient 2026-01-21 11:38:24.368886 | controller | -- openstack service list -f value -c Name -c Type\n logger.go:42: 11:38:20 | 2026-01-21 11:38:24.368893 | controller | watcher/5-cleanup-watcher | ++ grep -c '^watcher'\n logger.go:42: 11:38:23 | 2026-01-21 11:38:24.368901 | controller | watcher/5-cleanup-watcher | + '[' 0 == 0 ']'\n logger.go:42: 11:38:23 | watcher/5-cleanup-watcher 2026-01-21 11:38:24.368917 | controller | | test step completed 5-cleanup-watcher\n logger.go:42: 11:38:23 | watcher | 2026-01-21 11:38:24.368924 | controller | skipping kubernetes event logging\n=== CONT kuttl/harness/deps\n logger.go:42: 2026-01-21 11:38:24.368931 | controller | 11:38:24 | deps | Ignoring infra.yaml as it does not match file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n 2026-01-21 11:38:24.368939 | controller | \ logger.go:42: 11:38:24 | deps | Ignoring keystone.yaml as it does not match 2026-01-21 11:38:24.368950 | controller | file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n logger.go:42: 11:38:24 | 2026-01-21 11:38:24.368958 | controller | deps | Ignoring kustomization.yaml as it does not match file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n 2026-01-21 11:38:24.368965 | controller | \ logger.go:42: 11:38:24 | deps | Ignoring namespace.yaml as it does not match 2026-01-21 11:38:24.368972 | controller | file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n logger.go:42: 11:38:24 | 2026-01-21 11:38:24.368980 | controller | deps | Ignoring telemetry.yaml as it does not match file name regexp: ^(\\d+)-(?:[^\\.]+)(?:\\.yaml)?$\n 2026-01-21 11:38:24.368987 | controller | \ logger.go:42: 11:38:24 | deps | Skipping creation of user-supplied namespace: 2026-01-21 11:38:24.368995 | controller | watcher-kuttl-default\n logger.go:42: 11:38:24 | deps | skipping kubernetes event 2026-01-21 11:38:24.369002 | controller | logging\n=== NAME kuttl\n harness.go:406: run tests finished\n harness.go:514: 2026-01-21 11:38:24.369009 | controller | cleaning up\n harness.go:571: removing temp folder: \"\"\n--- FAIL: kuttl (1017.99s)\n 2026-01-21 11:38:24.369017 | controller | \ --- FAIL: kuttl/harness (0.00s)\n --- PASS: kuttl/harness/common (0.01s)\n 2026-01-21 11:38:24.369024 | controller | \ --- PASS: kuttl/harness/watcher-topology (80.92s)\n --- PASS: kuttl/harness/watcher-tls-certs-change 2026-01-21 11:38:24.369031 | controller | (51.51s)\n --- PASS: kuttl/harness/watcher-tls (159.28s)\n --- FAIL: 2026-01-21 11:38:24.369039 | controller | kuttl/harness/watcher-rmquser (300.24s)\n --- PASS: kuttl/harness/watcher-notification 2026-01-21 11:38:24.369046 | controller | (58.53s)\n --- PASS: kuttl/harness/watcher-cinder (179.74s)\n --- 2026-01-21 11:38:24.369053 | controller | PASS: kuttl/harness/watcher-api-scaling (75.82s)\n --- PASS: kuttl/harness/watcher 2026-01-21 11:38:24.369061 | controller | (111.88s)\n --- PASS: kuttl/harness/deps (0.01s)\nFAIL" 2026-01-21 11:38:24.369069 | controller | stdout_lines: 2026-01-21 11:38:24.369088 | controller | - oc kuttl test --v 1 --start-kind=false --config test/kuttl/test-suites/default/config.yaml 2026-01-21 11:38:24.369096 | controller | - === RUN kuttl 2026-01-21 11:38:24.369104 | controller | - ' harness.go:463: starting setup' 2026-01-21 11:38:24.369124 | controller | - ' harness.go:255: running tests using configured kubeconfig.' 2026-01-21 11:38:24.369132 | controller | - ' harness.go:278: Successful connection to cluster at: https://api.crc.testing:6443' 2026-01-21 11:38:24.369139 | controller | - ' harness.go:363: running tests' 2026-01-21 11:38:24.369147 | controller | - ' harness.go:75: going to run test suite with timeout of 300 seconds for each 2026-01-21 11:38:24.369154 | controller | step' 2026-01-21 11:38:24.369162 | controller | - ' harness.go:375: testsuite: test/kuttl/test-suites/default/ has 10 tests' 2026-01-21 11:38:24.369170 | controller | - === RUN kuttl/harness 2026-01-21 11:38:24.369178 | controller | - === RUN kuttl/harness/common 2026-01-21 11:38:24.369185 | controller | - === PAUSE kuttl/harness/common 2026-01-21 11:38:24.369192 | controller | - === RUN kuttl/harness/deps 2026-01-21 11:38:24.369200 | controller | - === PAUSE kuttl/harness/deps 2026-01-21 11:38:24.369207 | controller | - === RUN kuttl/harness/watcher 2026-01-21 11:38:24.369214 | controller | - === PAUSE kuttl/harness/watcher 2026-01-21 11:38:24.369222 | controller | - === RUN kuttl/harness/watcher-api-scaling 2026-01-21 11:38:24.369230 | controller | - === PAUSE kuttl/harness/watcher-api-scaling 2026-01-21 11:38:24.369239 | controller | - === RUN kuttl/harness/watcher-cinder 2026-01-21 11:38:24.369247 | controller | - === PAUSE kuttl/harness/watcher-cinder 2026-01-21 11:38:24.369259 | controller | - === RUN kuttl/harness/watcher-notification 2026-01-21 11:38:24.369269 | controller | - === PAUSE kuttl/harness/watcher-notification 2026-01-21 11:38:24.369277 | controller | - === RUN kuttl/harness/watcher-rmquser 2026-01-21 11:38:24.369286 | controller | - === PAUSE kuttl/harness/watcher-rmquser 2026-01-21 11:38:24.369298 | controller | - === RUN kuttl/harness/watcher-tls 2026-01-21 11:38:24.369306 | controller | - === PAUSE kuttl/harness/watcher-tls 2026-01-21 11:38:24.369313 | controller | - === RUN kuttl/harness/watcher-tls-certs-change 2026-01-21 11:38:24.369321 | controller | - === PAUSE kuttl/harness/watcher-tls-certs-change 2026-01-21 11:38:24.369328 | controller | - === RUN kuttl/harness/watcher-topology 2026-01-21 11:38:24.369336 | controller | - === PAUSE kuttl/harness/watcher-topology 2026-01-21 11:38:24.369343 | controller | - === CONT kuttl/harness/common 2026-01-21 11:38:24.369350 | controller | - ' logger.go:42: 11:21:26 | common | Ignoring cleanup-assert.yaml as it does not 2026-01-21 11:38:24.369358 | controller | match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.369366 | controller | - ' logger.go:42: 11:21:26 | common | Ignoring cleanup-errors.yaml as it does not 2026-01-21 11:38:24.369373 | controller | match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.369380 | controller | - ' logger.go:42: 11:21:26 | common | Ignoring cleanup-watcher.yaml as it does 2026-01-21 11:38:24.369388 | controller | not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.369395 | controller | - ' logger.go:42: 11:21:26 | common | Ignoring deploy-with-defaults.yaml as it 2026-01-21 11:38:24.369403 | controller | does not match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.369410 | controller | - ' logger.go:42: 11:21:26 | common | Skipping creation of user-supplied namespace: 2026-01-21 11:38:24.369418 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.369439 | controller | - ' logger.go:42: 11:21:26 | common | skipping kubernetes event logging' 2026-01-21 11:38:24.369447 | controller | - === CONT kuttl/harness/watcher-topology 2026-01-21 11:38:24.369454 | controller | - ' logger.go:42: 11:21:26 | watcher-topology | Skipping creation of user-supplied 2026-01-21 11:38:24.369462 | controller | namespace: watcher-kuttl-default' 2026-01-21 11:38:24.369469 | controller | - ' logger.go:42: 11:21:26 | watcher-topology/0-cleanup-watcher | starting test 2026-01-21 11:38:24.369477 | controller | step 0-cleanup-watcher' 2026-01-21 11:38:24.369484 | controller | - ' logger.go:42: 11:21:26 | watcher-topology/0-cleanup-watcher | test step completed 2026-01-21 11:38:24.369491 | controller | 0-cleanup-watcher' 2026-01-21 11:38:24.369499 | controller | - ' logger.go:42: 11:21:26 | watcher-topology/1-deploy-with-topology | starting 2026-01-21 11:38:24.369506 | controller | test step 1-deploy-with-topology' 2026-01-21 11:38:24.369514 | controller | - ' logger.go:42: 11:21:26 | watcher-topology/1-deploy-with-topology | Topology:watcher-kuttl-default/watcher-api 2026-01-21 11:38:24.369521 | controller | created' 2026-01-21 11:38:24.369529 | controller | - ' logger.go:42: 11:21:26 | watcher-topology/1-deploy-with-topology | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.369536 | controller | created' 2026-01-21 11:38:24.369544 | controller | - ' logger.go:42: 11:22:33 | watcher-topology/1-deploy-with-topology | test step 2026-01-21 11:38:24.369552 | controller | completed 1-deploy-with-topology' 2026-01-21 11:38:24.369560 | controller | - ' logger.go:42: 11:22:33 | watcher-topology/2-cleanup-watcher | starting test 2026-01-21 11:38:24.369568 | controller | step 2-cleanup-watcher' 2026-01-21 11:38:24.369575 | controller | - ' logger.go:42: 11:22:34 | watcher-topology/2-cleanup-watcher | running command: 2026-01-21 11:38:24.369582 | controller | [sh -c set -ex' 2026-01-21 11:38:24.369590 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.369597 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.369605 | controller | - ' ]' 2026-01-21 11:38:24.369612 | controller | - ' logger.go:42: 11:22:34 | watcher-topology/2-cleanup-watcher | + oc exec -n 2026-01-21 11:38:24.369623 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.369631 | controller | -c Type' 2026-01-21 11:38:24.369638 | controller | - ' logger.go:42: 11:22:34 | watcher-topology/2-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.369645 | controller | - ' logger.go:42: 11:22:38 | watcher-topology/2-cleanup-watcher | + ''['' 1 == 2026-01-21 11:38:24.369653 | controller | 0 '']''' 2026-01-21 11:38:24.369660 | controller | - ' logger.go:42: 11:22:39 | watcher-topology/2-cleanup-watcher | running command: 2026-01-21 11:38:24.369667 | controller | [sh -c set -ex' 2026-01-21 11:38:24.369675 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.369703 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.369729 | controller | - ' ]' 2026-01-21 11:38:24.369741 | controller | - ' logger.go:42: 11:22:39 | watcher-topology/2-cleanup-watcher | + oc exec -n 2026-01-21 11:38:24.369748 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.369756 | controller | -c Type' 2026-01-21 11:38:24.369764 | controller | - ' logger.go:42: 11:22:39 | watcher-topology/2-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.369772 | controller | - ' logger.go:42: 11:22:42 | watcher-topology/2-cleanup-watcher | + ''['' 0 == 2026-01-21 11:38:24.369780 | controller | 0 '']''' 2026-01-21 11:38:24.369787 | controller | - ' logger.go:42: 11:22:43 | watcher-topology/2-cleanup-watcher | running command: 2026-01-21 11:38:24.369795 | controller | [sh -c set -ex' 2026-01-21 11:38:24.369802 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.369809 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.369817 | controller | - ' ]' 2026-01-21 11:38:24.369824 | controller | - ' logger.go:42: 11:22:43 | watcher-topology/2-cleanup-watcher | + oc exec -n 2026-01-21 11:38:24.369831 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.369839 | controller | -c Type' 2026-01-21 11:38:24.369846 | controller | - ' logger.go:42: 11:22:43 | watcher-topology/2-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.369854 | controller | - ' logger.go:42: 11:22:46 | watcher-topology/2-cleanup-watcher | + ''['' 0 == 2026-01-21 11:38:24.369862 | controller | 0 '']''' 2026-01-21 11:38:24.369870 | controller | - ' logger.go:42: 11:22:46 | watcher-topology/2-cleanup-watcher | test step completed 2026-01-21 11:38:24.369877 | controller | 2-cleanup-watcher' 2026-01-21 11:38:24.369884 | controller | - ' logger.go:42: 11:22:46 | watcher-topology | skipping kubernetes event logging' 2026-01-21 11:38:24.369891 | controller | - === CONT kuttl/harness/watcher-tls-certs-change 2026-01-21 11:38:24.369899 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change | Skipping creation of user-supplied 2026-01-21 11:38:24.369907 | controller | namespace: watcher-kuttl-default' 2026-01-21 11:38:24.369915 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/0-cleanup-watcher | starting 2026-01-21 11:38:24.369922 | controller | test step 0-cleanup-watcher' 2026-01-21 11:38:24.369930 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/0-cleanup-watcher | test 2026-01-21 11:38:24.369937 | controller | step completed 0-cleanup-watcher' 2026-01-21 11:38:24.369945 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | starting 2026-01-21 11:38:24.369952 | controller | test step 1-deploy-with-tlse' 2026-01-21 11:38:24.369960 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | Certificate:watcher-kuttl-default/watcher-internal-svc 2026-01-21 11:38:24.369968 | controller | created' 2026-01-21 11:38:24.369975 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | Certificate:watcher-kuttl-default/watcher-public-svc 2026-01-21 11:38:24.369982 | controller | created' 2026-01-21 11:38:24.369990 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.370003 | controller | created' 2026-01-21 11:38:24.370010 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.370017 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.370025 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370032 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.370039 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.370047 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370055 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370062 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.370070 | controller | - ' exit 1' 2026-01-21 11:38:24.370077 | controller | - ' fi' 2026-01-21 11:38:24.370084 | controller | - ' ' 2026-01-21 11:38:24.370092 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370099 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.370107 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.370114 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370121 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370129 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.370136 | controller | - ' exit 1' 2026-01-21 11:38:24.370144 | controller | - ' fi' 2026-01-21 11:38:24.370151 | controller | - ' ]' 2026-01-21 11:38:24.370158 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.370165 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.370173 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.370180 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.370188 | controller | - ' logger.go:42: 11:22:47 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.370195 | controller | - ' logger.go:42: 11:22:48 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.370202 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.370210 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370234 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.370243 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.370250 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370257 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370265 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.370273 | controller | - ' exit 1' 2026-01-21 11:38:24.370280 | controller | - ' fi' 2026-01-21 11:38:24.370288 | controller | - ' ' 2026-01-21 11:38:24.370295 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370302 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.370310 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.370317 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370325 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370332 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.370339 | controller | - ' exit 1' 2026-01-21 11:38:24.370347 | controller | - ' fi' 2026-01-21 11:38:24.370354 | controller | - ' ]' 2026-01-21 11:38:24.370362 | controller | - ' logger.go:42: 11:22:48 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.370374 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.370381 | controller | - ' logger.go:42: 11:22:49 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.370389 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.370396 | controller | - ' logger.go:42: 11:22:49 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.370403 | controller | - ' logger.go:42: 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.370411 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.370418 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370425 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.370438 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.370445 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370453 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370460 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.370468 | controller | - ' exit 1' 2026-01-21 11:38:24.370475 | controller | - ' fi' 2026-01-21 11:38:24.370483 | controller | - ' ' 2026-01-21 11:38:24.370490 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370498 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.370505 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.370513 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370520 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370527 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.370535 | controller | - ' exit 1' 2026-01-21 11:38:24.370542 | controller | - ' fi' 2026-01-21 11:38:24.370550 | controller | - ' ]' 2026-01-21 11:38:24.370558 | controller | - ' logger.go:42: 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.370579 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.370587 | controller | - ' logger.go:42: 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.370594 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.370601 | controller | - ' logger.go:42: 11:22:50 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.370608 | controller | - ' logger.go:42: 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.370615 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.370622 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370630 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.370637 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.370645 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370652 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370659 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.370667 | controller | - ' exit 1' 2026-01-21 11:38:24.370674 | controller | - ' fi' 2026-01-21 11:38:24.370682 | controller | - ' ' 2026-01-21 11:38:24.370689 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370697 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.370705 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.370741 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370753 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370761 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.370768 | controller | - ' exit 1' 2026-01-21 11:38:24.370776 | controller | - ' fi' 2026-01-21 11:38:24.370783 | controller | - ' ]' 2026-01-21 11:38:24.370805 | controller | - ' logger.go:42: 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.370813 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.370820 | controller | - ' logger.go:42: 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.370828 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.370835 | controller | - ' logger.go:42: 11:22:51 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.370842 | controller | - ' logger.go:42: 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.370850 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.370858 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370865 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.370873 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.370880 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370888 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370895 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.370903 | controller | - ' exit 1' 2026-01-21 11:38:24.370910 | controller | - ' fi' 2026-01-21 11:38:24.370918 | controller | - ' ' 2026-01-21 11:38:24.370925 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.370933 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.370940 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.370948 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.370955 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.370963 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.370970 | controller | - ' exit 1' 2026-01-21 11:38:24.370978 | controller | - ' fi' 2026-01-21 11:38:24.370985 | controller | - ' ]' 2026-01-21 11:38:24.370993 | controller | - ' logger.go:42: 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.371000 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.371007 | controller | - ' logger.go:42: 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.371015 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.371022 | controller | - ' logger.go:42: 11:22:52 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.371030 | controller | - ' logger.go:42: 11:22:53 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.371038 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.371046 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371053 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.371061 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.371068 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371076 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371088 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.371100 | controller | - ' exit 1' 2026-01-21 11:38:24.371107 | controller | - ' fi' 2026-01-21 11:38:24.371115 | controller | - ' ' 2026-01-21 11:38:24.371122 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371130 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.371137 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.371144 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371152 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371159 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.371167 | controller | - ' exit 1' 2026-01-21 11:38:24.371175 | controller | - ' fi' 2026-01-21 11:38:24.371183 | controller | - ' ]' 2026-01-21 11:38:24.371192 | controller | - ' logger.go:42: 11:22:53 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.371200 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.371207 | controller | - ' logger.go:42: 11:22:54 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.371214 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.371222 | controller | - ' logger.go:42: 11:22:54 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.371229 | controller | - ' logger.go:42: 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.371237 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.371248 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371256 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.371263 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.371270 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371278 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371285 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.371292 | controller | - ' exit 1' 2026-01-21 11:38:24.371302 | controller | - ' fi' 2026-01-21 11:38:24.371323 | controller | - ' ' 2026-01-21 11:38:24.371331 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371338 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.371346 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.371353 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371360 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371368 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.371375 | controller | - ' exit 1' 2026-01-21 11:38:24.371383 | controller | - ' fi' 2026-01-21 11:38:24.371390 | controller | - ' ]' 2026-01-21 11:38:24.371398 | controller | - ' logger.go:42: 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.371405 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.371412 | controller | - ' logger.go:42: 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.371433 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.371442 | controller | - ' logger.go:42: 11:22:55 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.371449 | controller | - ' logger.go:42: 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.371456 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.371464 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371476 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.371484 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.371491 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371499 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371506 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.371513 | controller | - ' exit 1' 2026-01-21 11:38:24.371521 | controller | - ' fi' 2026-01-21 11:38:24.371528 | controller | - ' ' 2026-01-21 11:38:24.371618 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371631 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.371639 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.371646 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371653 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371661 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.371668 | controller | - ' exit 1' 2026-01-21 11:38:24.371676 | controller | - ' fi' 2026-01-21 11:38:24.371683 | controller | - ' ]' 2026-01-21 11:38:24.371690 | controller | - ' logger.go:42: 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.371698 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.371705 | controller | - ' logger.go:42: 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.371731 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.371743 | controller | - ' logger.go:42: 11:22:56 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.371751 | controller | - ' logger.go:42: 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.371758 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.371766 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371773 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.371781 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.371788 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371795 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371803 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.371810 | controller | - ' exit 1' 2026-01-21 11:38:24.371818 | controller | - ' fi' 2026-01-21 11:38:24.371825 | controller | - ' ' 2026-01-21 11:38:24.371832 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371839 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.371847 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.371854 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.371862 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.371869 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.371876 | controller | - ' exit 1' 2026-01-21 11:38:24.371884 | controller | - ' fi' 2026-01-21 11:38:24.371894 | controller | - ' ]' 2026-01-21 11:38:24.371901 | controller | - ' logger.go:42: 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.371909 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.371917 | controller | - ' logger.go:42: 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.371924 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.371937 | controller | - ' logger.go:42: 11:22:57 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.371958 | controller | - ' logger.go:42: 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.371966 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.371973 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.371981 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.371988 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.371996 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372003 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372010 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.372018 | controller | - ' exit 1' 2026-01-21 11:38:24.372025 | controller | - ' fi' 2026-01-21 11:38:24.372033 | controller | - ' ' 2026-01-21 11:38:24.372040 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372047 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.372055 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.372063 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372070 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372078 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.372085 | controller | - ' exit 1' 2026-01-21 11:38:24.372093 | controller | - ' fi' 2026-01-21 11:38:24.372101 | controller | - ' ]' 2026-01-21 11:38:24.372109 | controller | - ' logger.go:42: 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.372117 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.372125 | controller | - ' logger.go:42: 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.372132 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.372140 | controller | - ' logger.go:42: 11:22:59 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.372156 | controller | - ' logger.go:42: 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.372164 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.372171 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372178 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.372185 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.372193 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372200 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372208 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.372215 | controller | - ' exit 1' 2026-01-21 11:38:24.372222 | controller | - ' fi' 2026-01-21 11:38:24.372231 | controller | - ' ' 2026-01-21 11:38:24.372240 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372247 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.372255 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.372264 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372273 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372282 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.372290 | controller | - ' exit 1' 2026-01-21 11:38:24.372298 | controller | - ' fi' 2026-01-21 11:38:24.372310 | controller | - ' ]' 2026-01-21 11:38:24.372317 | controller | - ' logger.go:42: 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.372325 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.372332 | controller | - ' logger.go:42: 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.372339 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.372347 | controller | - ' logger.go:42: 11:23:00 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.372354 | controller | - ' logger.go:42: 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.372361 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.372369 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372376 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.372383 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.372391 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372398 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372406 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.372413 | controller | - ' exit 1' 2026-01-21 11:38:24.372421 | controller | - ' fi' 2026-01-21 11:38:24.372428 | controller | - ' ' 2026-01-21 11:38:24.372436 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372443 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.372451 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.372458 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372466 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372487 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.372495 | controller | - ' exit 1' 2026-01-21 11:38:24.372502 | controller | - ' fi' 2026-01-21 11:38:24.372510 | controller | - ' ]' 2026-01-21 11:38:24.372517 | controller | - ' logger.go:42: 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.372525 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.372532 | controller | - ' logger.go:42: 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.372540 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.372547 | controller | - ' logger.go:42: 11:23:01 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.372555 | controller | - ' logger.go:42: 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.372564 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.372577 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372585 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.372592 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.372600 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372608 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372615 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.372624 | controller | - ' exit 1' 2026-01-21 11:38:24.372633 | controller | - ' fi' 2026-01-21 11:38:24.372640 | controller | - ' ' 2026-01-21 11:38:24.372648 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372655 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.372669 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.372676 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372684 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372691 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.372699 | controller | - ' exit 1' 2026-01-21 11:38:24.372706 | controller | - ' fi' 2026-01-21 11:38:24.372743 | controller | - ' ]' 2026-01-21 11:38:24.372751 | controller | - ' logger.go:42: 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.372759 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.372767 | controller | - ' logger.go:42: 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.372774 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.372782 | controller | - ' logger.go:42: 11:23:03 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.372789 | controller | - ' logger.go:42: 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.372796 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.372803 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372811 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.372818 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.372825 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372832 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372840 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.372847 | controller | - ' exit 1' 2026-01-21 11:38:24.372855 | controller | - ' fi' 2026-01-21 11:38:24.372862 | controller | - ' ' 2026-01-21 11:38:24.372869 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.372877 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.372884 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.372892 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.372899 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.372906 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.372914 | controller | - ' exit 1' 2026-01-21 11:38:24.372921 | controller | - ' fi' 2026-01-21 11:38:24.372936 | controller | - ' ]' 2026-01-21 11:38:24.372945 | controller | - ' logger.go:42: 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.372952 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.372964 | controller | - ' logger.go:42: 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | Error 2026-01-21 11:38:24.372972 | controller | from server (NotFound): pods "watcher-kuttl-api-0" not found' 2026-01-21 11:38:24.372979 | controller | - ' logger.go:42: 11:23:04 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.372987 | controller | - ' logger.go:42: 11:23:05 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.372994 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.373002 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.373009 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.373017 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.373039 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.373048 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.373060 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.373068 | controller | - ' exit 1' 2026-01-21 11:38:24.373075 | controller | - ' fi' 2026-01-21 11:38:24.373083 | controller | - ' ' 2026-01-21 11:38:24.373090 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.373097 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.373105 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.373113 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.373120 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.373127 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.373135 | controller | - ' exit 1' 2026-01-21 11:38:24.373142 | controller | - ' fi' 2026-01-21 11:38:24.373150 | controller | - ' ]' 2026-01-21 11:38:24.373157 | controller | - ' logger.go:42: 11:23:05 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.373164 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.373172 | controller | - ' logger.go:42: 11:23:06 | watcher-tls-certs-change/1-deploy-with-tlse | error: 2026-01-21 11:38:24.373180 | controller | unable to upgrade connection: container not found ("watcher-api")' 2026-01-21 11:38:24.373189 | controller | - ' logger.go:42: 11:23:06 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.373197 | controller | - ' logger.go:42: 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.373205 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.373212 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.373219 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.373227 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.373235 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.373242 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.373249 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.373256 | controller | - ' exit 1' 2026-01-21 11:38:24.373264 | controller | - ' fi' 2026-01-21 11:38:24.373271 | controller | - ' ' 2026-01-21 11:38:24.373279 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.373286 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.373310 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.373317 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.373324 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.373331 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.373339 | controller | - ' exit 1' 2026-01-21 11:38:24.373350 | controller | - ' fi' 2026-01-21 11:38:24.373358 | controller | - ' ]' 2026-01-21 11:38:24.373365 | controller | - ' logger.go:42: 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.373373 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.373380 | controller | - ' logger.go:42: 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | error: 2026-01-21 11:38:24.373388 | controller | unable to upgrade connection: container not found ("watcher-api")' 2026-01-21 11:38:24.373395 | controller | - ' logger.go:42: 11:23:07 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=' 2026-01-21 11:38:24.373403 | controller | - ' logger.go:42: 11:23:08 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.373410 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.373425 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.373433 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.373440 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.373447 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.373455 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.373462 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.373469 | controller | - ' exit 1' 2026-01-21 11:38:24.373476 | controller | - ' fi' 2026-01-21 11:38:24.373484 | controller | - ' ' 2026-01-21 11:38:24.373491 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.373499 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.373506 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.373514 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.373521 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.373528 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.373536 | controller | - ' exit 1' 2026-01-21 11:38:24.373543 | controller | - ' fi' 2026-01-21 11:38:24.373551 | controller | - ' ]' 2026-01-21 11:38:24.373558 | controller | - ' logger.go:42: 11:23:08 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.373566 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.373597 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=''-----BEGIN 2026-01-21 11:38:24.373606 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.373613 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.373622 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.373629 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.373637 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.373644 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.373651 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.373659 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.373666 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.373674 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.373686 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.373694 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.373701 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.373731 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.373742 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.373750 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.373757 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.373765 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.373772 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.373779 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.373787 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.373795 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.373802 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.373810 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.373818 | controller | --decode' 2026-01-21 11:38:24.373825 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.373833 | controller | get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.373840 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert=''-----BEGIN 2026-01-21 11:38:24.373847 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.373855 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.373862 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.373869 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.373889 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.373897 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.373904 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.373912 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.373919 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.373927 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.373934 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.373942 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.373958 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.373965 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.373972 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.373980 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.373987 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.373994 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.374002 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.374009 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.374016 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.374024 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.374031 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.374039 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.374046 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.374054 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.374061 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.374069 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.374076 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.374084 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.374091 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.374098 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.374106 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.374113 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.374132 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.374139 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.374147 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.374158 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.374174 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.374182 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.374189 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.374196 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.374204 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.374211 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.374219 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.374227 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.374237 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.374245 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.374253 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.374261 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.374270 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.374278 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.374287 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.374295 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.374302 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.374309 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.374317 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.374324 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.374331 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.374339 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.374347 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.374359 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.374371 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.374391 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.374398 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.374406 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.374414 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.374421 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.374428 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.374440 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.374448 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.374455 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert=''-----BEGIN 2026-01-21 11:38:24.374462 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.374470 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.374478 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.374485 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.374492 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.374500 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.374507 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.374515 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.374522 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.374530 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.374538 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.374545 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.374553 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.374561 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.374596 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.374610 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.374618 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.374625 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.374637 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.374645 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.374652 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.374659 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.374670 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.374690 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.374698 | controller | get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.374706 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.374764 | controller | --decode' 2026-01-21 11:38:24.374772 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert=''-----BEGIN 2026-01-21 11:38:24.374779 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.374787 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.374794 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.374802 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.374809 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.374817 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.374824 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.374832 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.374839 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.374847 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.374854 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.374861 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.374869 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.374876 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.374888 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.374896 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.374903 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.374910 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.374917 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.374925 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.374937 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.374944 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.374951 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.374959 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.374967 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.374974 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.374982 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.374989 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.375010 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.375018 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.375025 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.375033 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.375040 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.375048 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.375055 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.375062 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.375070 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.375077 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.375084 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.375098 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.375105 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.375113 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.375120 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.375128 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.375135 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.375143 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.375150 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.375158 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.375165 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.375177 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.375185 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.375192 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.375200 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.375207 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.375214 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.375222 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.375229 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.375255 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.375263 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.375270 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.375278 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.375286 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.375294 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.375306 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.375314 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.375321 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.375329 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.375337 | controller | - ' logger.go:42: 11:23:09 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.375345 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.375353 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.375360 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.375368 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.375375 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.375383 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.375390 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.375397 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.375405 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.375412 | controller | - ' exit 1' 2026-01-21 11:38:24.375420 | controller | - ' fi' 2026-01-21 11:38:24.375427 | controller | - ' ' 2026-01-21 11:38:24.375435 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.375442 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.375450 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.375457 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.375464 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.375472 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.375479 | controller | - ' exit 1' 2026-01-21 11:38:24.375487 | controller | - ' fi' 2026-01-21 11:38:24.375494 | controller | - ' ]' 2026-01-21 11:38:24.375502 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.375509 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.375517 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=''-----BEGIN 2026-01-21 11:38:24.375524 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.375531 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.375539 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.375547 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.375555 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.375568 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.375575 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.375587 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.375605 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.375613 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.375621 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.375629 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.375636 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.375643 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.375651 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.375658 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.375666 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.375673 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.375680 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.375688 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.375695 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.375703 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.375734 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.375746 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.375754 | controller | get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.375762 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.375769 | controller | --decode' 2026-01-21 11:38:24.375777 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert=''-----BEGIN 2026-01-21 11:38:24.375785 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.375792 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.375799 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.375807 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.375814 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.375822 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.375834 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.375851 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.375859 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.375866 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.375874 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.375881 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.375899 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.375907 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.375914 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.375922 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.375929 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.375936 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.375944 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.375951 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.375959 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.375966 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.375974 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.375981 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.375989 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.375996 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.376003 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.376011 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.376018 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.376086 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.376095 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.376107 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.376115 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.376126 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.376134 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.376141 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.376149 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.376156 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.376164 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.376172 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.376179 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.376188 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.376210 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.376218 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.376225 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.376232 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.376240 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.376247 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.376254 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.376261 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.376268 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.376276 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.376283 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.376291 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.376298 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.376315 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.376322 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.376333 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.376341 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.376348 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.376355 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.376363 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.376370 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.376378 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.376385 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.376392 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.376400 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.376407 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.376414 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.376421 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.376432 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.376439 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert=''-----BEGIN 2026-01-21 11:38:24.376447 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.376454 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.376471 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.376479 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.376487 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.376494 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.376501 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.376508 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.376516 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.376528 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.376535 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.376542 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.376550 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.376562 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.376570 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.376577 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.376585 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.376592 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.376599 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.376607 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.376614 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.376621 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.376629 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.376636 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.376644 | controller | get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.376652 | controller | - ' logger.go:42: 11:23:11 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.376659 | controller | --decode' 2026-01-21 11:38:24.376666 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert=''-----BEGIN 2026-01-21 11:38:24.376674 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.376681 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.376689 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.376696 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.376703 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.376735 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.376747 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.376759 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.376767 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.376774 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.376781 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.376789 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.376796 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.376803 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.376811 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.376822 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.376829 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.376837 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.376844 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.376852 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.376859 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.376867 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.376874 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.376881 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.376889 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.376896 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.376904 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.376911 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.376919 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.376926 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.376933 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.376941 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.376948 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.376959 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.376966 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.376974 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.376991 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.376998 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.377006 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.377013 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.377020 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.377054 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.377061 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.377068 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.377076 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.377083 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.377091 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.377098 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.377105 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.377113 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.377120 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.377127 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.377135 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.377142 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.377149 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.377157 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.377164 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.377176 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.377183 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.377191 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.377198 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.377205 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.377213 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.377220 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.377238 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.377255 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.377263 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.377272 | controller | - ' logger.go:42: 11:23:12 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.377280 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.377289 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.377296 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.377308 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.377315 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.377323 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.377330 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.377337 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.377345 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.377352 | controller | - ' exit 1' 2026-01-21 11:38:24.377359 | controller | - ' fi' 2026-01-21 11:38:24.377367 | controller | - ' ' 2026-01-21 11:38:24.377374 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.377382 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.377389 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.377397 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.377404 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.377412 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.377419 | controller | - ' exit 1' 2026-01-21 11:38:24.377427 | controller | - ' fi' 2026-01-21 11:38:24.377434 | controller | - ' ]' 2026-01-21 11:38:24.377441 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.377449 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.377457 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=''-----BEGIN 2026-01-21 11:38:24.377470 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.377478 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.377485 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.377493 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.377500 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.377508 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.377515 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.377522 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.377530 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.377537 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.377545 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.377552 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.377560 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.377567 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.377574 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.377594 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.377602 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.377609 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.377616 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.377624 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.377631 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.377639 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.377647 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.377654 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.377662 | controller | get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.377673 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.377681 | controller | --decode' 2026-01-21 11:38:24.377688 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert=''-----BEGIN 2026-01-21 11:38:24.377695 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.377702 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.377734 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.377742 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.377749 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.377757 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.377764 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.377771 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.377779 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.377786 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.377794 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.377801 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.377809 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.377816 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.377823 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.377831 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.377838 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.377845 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.377853 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.377871 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.377879 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.377886 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.377894 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.377901 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.377914 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.377921 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.377929 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.377936 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.377948 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.377955 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.377962 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.377970 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.377977 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.377984 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.377992 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.378000 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.378007 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.378015 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.378023 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.378030 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.378038 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.378045 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.378053 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.378060 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.378068 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.378075 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.378083 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.378091 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.378098 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.378110 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.378117 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.378135 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.378149 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.378156 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.378164 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.378172 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.378180 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.378189 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.378197 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.378204 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.378212 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.378219 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.378226 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.378234 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.378241 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.378249 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.378256 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.378264 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.378271 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.378279 | controller | - ' logger.go:42: 11:23:13 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.378286 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.378294 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert=''-----BEGIN 2026-01-21 11:38:24.378301 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.378309 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.378321 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.378329 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.378336 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.378343 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.378351 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.378358 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.378370 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.378388 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.378396 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.378403 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.378411 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.378418 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.378477 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.378484 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.378491 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.378499 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.378507 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.378515 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.378522 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.378530 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.378537 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.378545 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.378553 | controller | get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.378560 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.378591 | controller | --decode' 2026-01-21 11:38:24.378599 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert=''-----BEGIN 2026-01-21 11:38:24.378612 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.378619 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.378627 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.378634 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.378641 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.378649 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.378656 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.378663 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.378671 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.378678 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.378691 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.378700 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.378707 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.378750 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.378760 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.378767 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.378774 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.378782 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.378789 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.378796 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.378804 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.378811 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.378819 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.378826 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.378834 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.378841 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.378853 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.378861 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.378868 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.378875 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.378883 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.378890 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.378898 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.378905 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.378913 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.378920 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.378932 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.378939 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.378947 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.378954 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.378962 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.378969 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.378977 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.378998 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.379006 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.379014 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.379021 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.379029 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.379036 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.379044 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.379059 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.379067 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.379074 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.379082 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.379089 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.379096 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.379104 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.379111 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.379119 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.379126 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.379138 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.379146 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.379154 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.379161 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.379168 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.379175 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.379183 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.379190 | controller | - ' logger.go:42: 11:23:14 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.379198 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.379205 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.379213 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.379221 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.379228 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.379238 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.379245 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.379253 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.379261 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.379270 | controller | - ' exit 1' 2026-01-21 11:38:24.379284 | controller | - ' fi' 2026-01-21 11:38:24.379292 | controller | - ' ' 2026-01-21 11:38:24.379309 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.379317 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.379325 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.379332 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.379340 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.379347 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.379355 | controller | - ' exit 1' 2026-01-21 11:38:24.379362 | controller | - ' fi' 2026-01-21 11:38:24.379369 | controller | - ' ]' 2026-01-21 11:38:24.379377 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.379384 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.379392 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=''-----BEGIN 2026-01-21 11:38:24.379400 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.379407 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.379415 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.379422 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.379430 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.379437 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.379444 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.379452 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.379459 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.379466 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.379474 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.379481 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.379489 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.379496 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.379504 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.379511 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.379523 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.379535 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.379542 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.379550 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.379557 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.379565 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.379572 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.379580 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.379587 | controller | --decode' 2026-01-21 11:38:24.379595 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.379615 | controller | get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.379623 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert=''-----BEGIN 2026-01-21 11:38:24.379630 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.379638 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.379646 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.379653 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.379661 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.379668 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.379675 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.379683 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.379690 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.379697 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.379705 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.379728 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.379740 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.379748 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.379755 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.379762 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.379774 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.379782 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.379789 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.379797 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.379804 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.379817 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.379824 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.379832 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.379839 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.379847 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.379854 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.379862 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.379869 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.379876 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.379893 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.379900 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.379908 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.379915 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.379922 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.379930 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.379937 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.379945 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.379952 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.379960 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.379968 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.379980 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.379987 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.379995 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.380006 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.380014 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.380021 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.380029 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.380036 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.380043 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.380051 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.380058 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.380066 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.380073 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.380081 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.380088 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.380096 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.380103 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.380118 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.380126 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.380133 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.380141 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.380148 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.380155 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.380163 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.380175 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.380184 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.380192 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.380200 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.380207 | controller | - ' logger.go:42: 11:23:15 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.380215 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.380222 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert=''-----BEGIN 2026-01-21 11:38:24.380230 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.380238 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.380245 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.380257 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.380264 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.380272 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.380279 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.380287 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.380294 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.380301 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.380308 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.380315 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.380323 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.380330 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.380337 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.380345 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.380352 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.380369 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.380377 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.380390 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.380398 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.380405 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.380413 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.380424 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.380431 | controller | get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.380439 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.380446 | controller | --decode' 2026-01-21 11:38:24.380454 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert=''-----BEGIN 2026-01-21 11:38:24.380461 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.380468 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.380476 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.380483 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.380490 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.380501 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.380509 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.380516 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.380524 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.380532 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.380539 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.380547 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.380554 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.380561 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.380569 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.380576 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.380583 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.380591 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.380606 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.380613 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.380621 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.380641 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.380649 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.380656 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.380664 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.380671 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.380678 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.380686 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.380694 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.380701 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.380709 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.380742 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.380751 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.380758 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.380765 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.380772 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.380780 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.380787 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.380794 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.380802 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.380809 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.380817 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.380824 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.380836 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.380844 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.380852 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.380860 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.380867 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.380875 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.380882 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.380889 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.380896 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.380914 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.380922 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.380929 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.380941 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.380949 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.380956 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.380964 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.380971 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.380978 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.380986 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.380993 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.381001 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.381008 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.381016 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.381023 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.381035 | controller | - ' logger.go:42: 11:23:16 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.381042 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.381049 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.381057 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.381064 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.381072 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.381079 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.381087 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.381094 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.381102 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.381109 | controller | - ' exit 1' 2026-01-21 11:38:24.381117 | controller | - ' fi' 2026-01-21 11:38:24.381124 | controller | - ' ' 2026-01-21 11:38:24.381132 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.381139 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.381146 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.381154 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.381161 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.381168 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.381176 | controller | - ' exit 1' 2026-01-21 11:38:24.381183 | controller | - ' fi' 2026-01-21 11:38:24.381190 | controller | - ' ]' 2026-01-21 11:38:24.381198 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.381205 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.381212 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=''-----BEGIN 2026-01-21 11:38:24.381220 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.381227 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.381237 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.381254 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.381263 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.381272 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.381281 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.381290 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.381297 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.381304 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.381312 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.381328 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.381336 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.381343 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.381351 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.381358 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.381365 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.381373 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.381380 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.381388 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.381395 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.381403 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.381410 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.381418 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.381425 | controller | get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.381437 | controller | - ' logger.go:42: 11:23:17 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.381444 | controller | --decode' 2026-01-21 11:38:24.381452 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert=''-----BEGIN 2026-01-21 11:38:24.381459 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.381466 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.381474 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.381481 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.381489 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.381496 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.381504 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.381520 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.381529 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.381536 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.381548 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.381556 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.381564 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.381572 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.381579 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.381587 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.381594 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.381601 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.381609 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.381616 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.381628 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.381635 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.381643 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.381650 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.381658 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.381665 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.381673 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.381681 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.381688 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.381695 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.381703 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.381710 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.381739 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.381747 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.381754 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.381767 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.381774 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.381793 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.381801 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.381814 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.381822 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.381829 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.381836 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.381844 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.381852 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.381859 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.381867 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.381874 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.381882 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.381889 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.381897 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.381904 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.381912 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.381919 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.381926 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.381933 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.381941 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.381948 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.381956 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.381968 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.381975 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.381983 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.381990 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.382002 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.382011 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.382018 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.382039 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.382048 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.382056 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.382074 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.382082 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.382090 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert=''-----BEGIN 2026-01-21 11:38:24.382097 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.382105 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.382113 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.382120 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.382128 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.382136 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.382143 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.382151 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.382159 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.382166 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.382174 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.382181 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.382196 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.382203 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.382210 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.382218 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.382225 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.382232 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.382240 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.382247 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.382254 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.382262 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.382269 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.382282 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.382289 | controller | --decode' 2026-01-21 11:38:24.382297 | controller | - ' logger.go:42: 11:23:18 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.382304 | controller | get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.382312 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert=''-----BEGIN 2026-01-21 11:38:24.382319 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.382337 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.382345 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.382352 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.382360 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.382367 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.382374 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.382382 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.382389 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.382396 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.382404 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.382415 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.382422 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.382430 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.382437 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.382444 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.382452 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.382459 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.382466 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.382473 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.382481 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.382488 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.382495 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.382503 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.382510 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.382518 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.382531 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.382538 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.382545 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.382553 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.382561 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.382600 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.382611 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.382618 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.382626 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.382633 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.382645 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.382653 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.382660 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.382667 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.382675 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.382682 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.382689 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.382697 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.382704 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.382712 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.382739 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.382747 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.382755 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.382762 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.382773 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.382781 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.382789 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.382796 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.382803 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.382811 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.382818 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.382826 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.382833 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.382840 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.382863 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.382871 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.382878 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.382886 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.382894 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.382901 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.382909 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.382916 | controller | - ' logger.go:42: 11:23:19 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.382924 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.382932 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | running 2026-01-21 11:38:24.382939 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.382947 | controller | - ' public_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.382954 | controller | cat /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.382961 | controller | - ' public_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc 2026-01-21 11:38:24.382968 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.382976 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.382983 | controller | - ' if [ "${public_svc_cert}" != "${public_secret_cert}" ]; then' 2026-01-21 11:38:24.382991 | controller | - ' exit 1' 2026-01-21 11:38:24.382998 | controller | - ' fi' 2026-01-21 11:38:24.383005 | controller | - ' ' 2026-01-21 11:38:24.383012 | controller | - ' internal_svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 2026-01-21 11:38:24.383019 | controller | cat /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.383026 | controller | - ' internal_secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc 2026-01-21 11:38:24.383034 | controller | -o jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.383041 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.383049 | controller | - ' if [ "${internal_svc_cert}" != "${internal_secret_cert}" ]; then' 2026-01-21 11:38:24.383056 | controller | - ' exit 1' 2026-01-21 11:38:24.383064 | controller | - ' fi' 2026-01-21 11:38:24.383071 | controller | - ' ]' 2026-01-21 11:38:24.383079 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.383086 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.383093 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | + public_svc_cert=''-----BEGIN 2026-01-21 11:38:24.383101 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.383108 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.383115 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.383122 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.383134 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.383142 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.383154 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.383161 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.383168 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.383175 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.383183 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.383202 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.383211 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.383219 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.383227 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.383237 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.383245 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.383253 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.383262 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.383271 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.383279 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.383289 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.383297 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.383305 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.383312 | controller | get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.383320 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.383328 | controller | --decode' 2026-01-21 11:38:24.383335 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | + public_secret_cert=''-----BEGIN 2026-01-21 11:38:24.383343 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.383351 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.383359 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.383371 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.383379 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.383387 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.383394 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.383402 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.383414 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.383422 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.383430 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.383437 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.383445 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.383452 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.383460 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.383478 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.383488 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.383496 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.383503 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.383511 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.383518 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.383525 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.383532 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.383539 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.383547 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.383560 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.383567 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.383574 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.383586 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.383593 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.383600 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.383607 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.383614 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.383620 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.383634 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.383642 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.383650 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.383657 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.383665 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.383672 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.383680 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.383687 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.383695 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.383703 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.383710 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.383749 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.383768 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.383776 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.383783 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.383791 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.383798 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.383805 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.383818 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.383825 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.383832 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.383840 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.383847 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.383855 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.383867 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.383875 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.383882 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.383890 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.383898 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.383906 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.383913 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.383920 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.383928 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.383935 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.383943 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.383951 | controller | - ' logger.go:42: 11:23:20 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.383958 | controller | rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.383966 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_svc_cert=''-----BEGIN 2026-01-21 11:38:24.383973 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.383981 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.383988 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.383996 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.384003 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.384021 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.384033 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.384041 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.384049 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.384056 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.384064 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.384072 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.384079 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.384087 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.384099 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.384106 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.384114 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.384121 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.384129 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.384136 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.384144 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.384151 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.384159 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.384166 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | ++ base64 2026-01-21 11:38:24.384174 | controller | --decode' 2026-01-21 11:38:24.384181 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | ++ oc 2026-01-21 11:38:24.384189 | controller | get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.384196 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | + internal_secret_cert=''-----BEGIN 2026-01-21 11:38:24.384204 | controller | CERTIFICATE-----' 2026-01-21 11:38:24.384212 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.384220 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.384227 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.384235 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.384247 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.384254 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.384262 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.384270 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.384285 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.384293 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.384300 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.384308 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.384315 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.384323 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.384330 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.384342 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.384350 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.384358 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.384365 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.384372 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.384380 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.384387 | controller | CERTIFICATE-----''' 2026-01-21 11:38:24.384395 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | + ''['' 2026-01-21 11:38:24.384402 | controller | ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.384410 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.384418 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.384425 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.384432 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.384440 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.384452 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.384459 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.384467 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.384474 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.384481 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.384489 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.384497 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.384505 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.384512 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.384529 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.384538 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.384545 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.384558 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.384566 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.384573 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.384581 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.384589 | controller | CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.384596 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MIIDtjCCAh6gAwIBAgIQJY8kGKPKL6pExqzlkcSGZzANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.384604 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMjQ4WhcNMzEw' 2026-01-21 11:38:24.384611 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MTIwMTEyMjQ4WjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4DvR' 2026-01-21 11:38:24.384619 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lKai6OqSQiJRfyzB7x121ugJajZKB1PSb2riAuzgQ3F7yXVfOjr+GQ6jT0t6MkD0' 2026-01-21 11:38:24.384626 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | XOyYLWLHtzvLczjyZY6ARyZHY65mKDfnztbgawZFVJQhmSJy3EFSkTI9dGLJd620' 2026-01-21 11:38:24.384633 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | lLFRG70q3J+S4ufm9TTyTzDcl5UaAacmEjbwYL8f4L4ub7VOOXoMGaZpQ0uERMW5' 2026-01-21 11:38:24.384641 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | b7/KviOmsOhVQbmij2sx4lWvPQWEdpZ04bDfw+ov0qPcrN2MNW4He1ObZXcs4GyO' 2026-01-21 11:38:24.384653 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Rrj7JmF6Bu7t6NxH5FAADC7mwjCE90NfvwdhGaWYVeeSJB2SnAzY34oijOjOyiO9' 2026-01-21 11:38:24.384661 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | SVN1UIDlrehPaHfoyQIDAQABo4GRMIGOMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.384669 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.384676 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | rbDBcPZ3k3kmpm+iMDgGA1UdEQEB/wQuMCyCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.384684 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Y2hlci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAEPCAVWKX' 2026-01-21 11:38:24.384691 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | Aw7FZIoWaCbOFx40bY80YB40ItfQwDaoQ3ApmMf4tfb0ib7d3Q1S4CnxRvAMhOSO' 2026-01-21 11:38:24.384699 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | UbMGC3NnHvIY+sAe4rIYzR9fBprAueMR04A096YH7LqA3/iKePMPh51MRmUj2SWO' 2026-01-21 11:38:24.384706 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | BewTGMZN/UZFaZCA8B+j8nnSAoenouDz8mNIcWjoCxLj7SKF9ik9Z69UhR3fx9Ue' 2026-01-21 11:38:24.384729 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | gIvc6nWj51TSJTmPw51G0bHN/cdEu+SobFU1GJyMjyvGSjCrACYRSmrclxMPP1n2' 2026-01-21 11:38:24.384741 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | VkbfmEnuRpG2GLaAMNOz+3oMXENVMaNsKmL0Z3xNIAaojb0kMtCPBqhrs979V2cN' 2026-01-21 11:38:24.384748 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | sO3qzHYodf6MoxfFZP7OHcJ0G+2b1ZMhGmjOXF6jdF5tRMrHoIHqM2KExYc8YAvz' 2026-01-21 11:38:24.384755 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | j88N9OuXCAkYJHBJ9dJOvlsbVgVPJGqxRVJFCLBVxoxHK122imDR2Gaf0N1hdcSx' 2026-01-21 11:38:24.384767 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | MSh9rGfiaeCTExXBgYvidEHM8nq3oZwvEOkHBLASIJBbPpqTcuMifHZJ' 2026-01-21 11:38:24.384774 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | -----END 2026-01-21 11:38:24.384781 | controller | CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.384797 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/1-deploy-with-tlse | test 2026-01-21 11:38:24.384805 | controller | step completed 1-deploy-with-tlse' 2026-01-21 11:38:24.384812 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.384819 | controller | | starting test step 2-change-public-svc-certificate' 2026-01-21 11:38:24.384825 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.384832 | controller | | running command: [sh -c oc patch Certificate -n $NAMESPACE watcher-public-svc 2026-01-21 11:38:24.384839 | controller | --type=''json'' -p=''[{"op": "replace", "path": "/spec/dnsNames", "value":[''watcher-public.watcher-kuttl-default.svc'', 2026-01-21 11:38:24.384846 | controller | ''watcher-public.watcher-kuttl-default.svc.cluster.local'']}]''' 2026-01-21 11:38:24.384854 | controller | - ' ]' 2026-01-21 11:38:24.384862 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.384869 | controller | | certificate.cert-manager.io/watcher-public-svc patched' 2026-01-21 11:38:24.384877 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.384885 | controller | | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.384892 | controller | - ' svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat 2026-01-21 11:38:24.384904 | controller | /etc/pki/tls/certs/public.crt)' 2026-01-21 11:38:24.384937 | controller | - ' secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-public-svc -o jsonpath=''{.data.tls\.crt}'' 2026-01-21 11:38:24.384946 | controller | | base64 --decode)' 2026-01-21 11:38:24.384953 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.384961 | controller | - ' if [ "${svc_cert}" != "${secret_cert}" ]; then' 2026-01-21 11:38:24.384968 | controller | - ' exit 1' 2026-01-21 11:38:24.384976 | controller | - ' fi' 2026-01-21 11:38:24.384984 | controller | - ' ]' 2026-01-21 11:38:24.384991 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.384999 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/public.crt' 2026-01-21 11:38:24.385006 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385014 | controller | | + svc_cert=''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.385021 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385028 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.385036 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385044 | controller | | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.385056 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385064 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.385071 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385079 | controller | | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.385086 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385094 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.385101 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385109 | controller | | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.385117 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385124 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.385131 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385139 | controller | | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.385146 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385153 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.385161 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385168 | controller | | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.385176 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385185 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.385193 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385200 | controller | | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.385207 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385215 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.385222 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385230 | controller | | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.385252 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385261 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.385268 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385276 | controller | | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.385292 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385300 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.385307 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385314 | controller | | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.385322 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385329 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.385336 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385344 | controller | | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.385351 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385358 | controller | | -----END CERTIFICATE-----''' 2026-01-21 11:38:24.385366 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385373 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-public-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.385381 | controller | - ' logger.go:42: 11:23:21 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385388 | controller | | ++ base64 --decode' 2026-01-21 11:38:24.385395 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385402 | controller | | + secret_cert=''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.385410 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385417 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.385425 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385432 | controller | | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.385439 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385447 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.385454 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385461 | controller | | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.385469 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385480 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.385488 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385495 | controller | | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.385502 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385510 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.385517 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385525 | controller | | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.385532 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385545 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.385553 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385561 | controller | | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.385568 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385575 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.385583 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385590 | controller | | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.385598 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385605 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.385612 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385620 | controller | | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.385628 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385635 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.385652 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385660 | controller | | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.385668 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385675 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.385683 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385691 | controller | | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.385698 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385706 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.385728 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385740 | controller | | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.385747 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385754 | controller | | -----END CERTIFICATE-----''' 2026-01-21 11:38:24.385762 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385769 | controller | | + ''['' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.385777 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385784 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.385792 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385799 | controller | | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.385806 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385814 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.385822 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385829 | controller | | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.385836 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385843 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.385856 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385863 | controller | | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.385870 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385878 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.385885 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385893 | controller | | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.385900 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385907 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.385915 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385922 | controller | | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.385929 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385937 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.385944 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385952 | controller | | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.385959 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385967 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.385974 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.385987 | controller | | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.385997 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386005 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.386012 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386019 | controller | | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.386027 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386034 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.386041 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386049 | controller | | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.386067 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386075 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.386083 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386090 | controller | | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.386097 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386105 | controller | | -----END CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.386112 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386120 | controller | | MIIDszCCAhugAwIBAgIRALRAaudV90GDYH2itgQf2VkwDQYJKoZIhvcNAQEMBQAw' 2026-01-21 11:38:24.386127 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386134 | controller | | GDEWMBQGA1UEAxMNcm9vdGNhLXB1YmxpYzAeFw0yNjAxMjExMTIyNDdaFw0zMTAx' 2026-01-21 11:38:24.386142 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386153 | controller | | MjAxMTIyNDdaMAAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCng3GH' 2026-01-21 11:38:24.386161 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386168 | controller | | UWOv1NpImsNKc08O5uUqCfAa80B7KfBNA2mvzLpnJVa0NEG/xktGkefbBpJNjwCT' 2026-01-21 11:38:24.386176 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386183 | controller | | VgUbx0mM0JrnOl/JGr0KD7MiJZoB0nNKYWqjaMF61JcXQ1jSBvcsqPFc8GO9lhEt' 2026-01-21 11:38:24.386191 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386198 | controller | | zJunfD+9pGS+jNh/Ia70y2iMORzO8KDhdR/z3ExLp3sxCRSe148yO5iS0QeJ4yv+' 2026-01-21 11:38:24.386205 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386213 | controller | | 8tFQbY77I6lDGtemrazIzImjZaB2bLX6P49Y4xOwA4IBmjgwIrPU1HFm3TkqELoV' 2026-01-21 11:38:24.386220 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386228 | controller | | g3Nhg8giejvNuyiZNz82N1XV9IHIgH9tYoxln9YO99F3ciSS+9ltyIFU6ISoSLbH' 2026-01-21 11:38:24.386237 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386245 | controller | | EmfmkIZNQ4vGf2OTAgMBAAGjgY8wgYwwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM' 2026-01-21 11:38:24.386252 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386261 | controller | | MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUzqy2rEQSs8SK' 2026-01-21 11:38:24.386270 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386278 | controller | | jPSUqIq4oRKVwQ0wNgYDVR0RAQH/BCwwKoIod2F0Y2hlci1wdWJsaWMud2F0Y2hl' 2026-01-21 11:38:24.386287 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386294 | controller | | ci1rdXR0bC1kZWZhdWx0LnN2YzANBgkqhkiG9w0BAQwFAAOCAYEAk+YUWA13i2vG' 2026-01-21 11:38:24.386302 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386309 | controller | | PsjHYsgEdg8RaU5HZlfnZKzfcWER6y4w9fH46cC2z8SBs3adeHrFzJnzp1dedEdG' 2026-01-21 11:38:24.386317 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386324 | controller | | yWQTJi9zTOGdfTD2iia864TJqNBOUCK9yJdGDKLJAuGeU5I09pMrixdPo668Y3+q' 2026-01-21 11:38:24.386331 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386339 | controller | | nd7XioVT+pxPP7bHIoteUMF6+or+l6fiSxRh7MqOJTlTiO2fkrTD8m3o7bh4SVoN' 2026-01-21 11:38:24.386347 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386354 | controller | | 1u5foJASQJcItXQ52oNi77Qy6vTZRZ8nf7aoUSJ1zKvlXsMdvIeoz7UHmC8oJtkV' 2026-01-21 11:38:24.386361 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386369 | controller | | trTAiZ7u36k2C4GsPRfyetx6N3U4iPbADaU5dp5UR0RC8H0G/P9PzQrIf2NSFUQN' 2026-01-21 11:38:24.386376 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386384 | controller | | tKJ1ZxtztU7rVfB51aOd2vQilqEoTN27cIefz5YaiN92Q8da0ws36ZL3z6/1sTck' 2026-01-21 11:38:24.386391 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386399 | controller | | A77OaTnHj1rxqWdY93TotlAR+d9WP7GsZQnKqYuSMbT85euCwnNedqm/wW8Tcj1+' 2026-01-21 11:38:24.386406 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386413 | controller | | QTFANTH506dhGHXyuarw5eWnweynVaJuEpm7kjEY9B/NAfeVXu2R' 2026-01-21 11:38:24.386421 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386432 | controller | | -----END CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.386440 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/2-change-public-svc-certificate 2026-01-21 11:38:24.386457 | controller | | test step completed 2-change-public-svc-certificate' 2026-01-21 11:38:24.386464 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386472 | controller | | starting test step 3-change-internal-svc-certificate' 2026-01-21 11:38:24.386479 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386492 | controller | | running command: [sh -c oc patch Certificate -n $NAMESPACE watcher-internal-svc 2026-01-21 11:38:24.386499 | controller | --type=''json'' -p=''[{"op": "replace", "path": "/spec/dnsNames", "value":[''watcher-internal.watcher-kuttl-default.svc'', 2026-01-21 11:38:24.386507 | controller | ''watcher-internal.watcher-kuttl-default.svc.cluster.local'']}]''' 2026-01-21 11:38:24.386514 | controller | - ' ]' 2026-01-21 11:38:24.386522 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386530 | controller | | certificate.cert-manager.io/watcher-internal-svc patched' 2026-01-21 11:38:24.386537 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386545 | controller | | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.386552 | controller | - ' svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat 2026-01-21 11:38:24.386560 | controller | /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.386567 | controller | - ' secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc -o 2026-01-21 11:38:24.386600 | controller | jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.386608 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.386615 | controller | - ' if [ "${svc_cert}" != "${secret_cert}" ]; then' 2026-01-21 11:38:24.386623 | controller | - ' exit 1' 2026-01-21 11:38:24.386630 | controller | - ' fi' 2026-01-21 11:38:24.386638 | controller | - ' ]' 2026-01-21 11:38:24.386645 | controller | - ' logger.go:42: 11:23:22 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386653 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.386661 | controller | - ' logger.go:42: 11:23:24 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386668 | controller | | error: Internal error occurred: error executing command in container: container 2026-01-21 11:38:24.386676 | controller | is not created or running' 2026-01-21 11:38:24.386683 | controller | - ' logger.go:42: 11:23:24 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386691 | controller | | + svc_cert=' 2026-01-21 11:38:24.386698 | controller | - ' logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386706 | controller | | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.386731 | controller | - ' svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat 2026-01-21 11:38:24.386743 | controller | /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.386750 | controller | - ' secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc -o 2026-01-21 11:38:24.386758 | controller | jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.386765 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.386773 | controller | - ' if [ "${svc_cert}" != "${secret_cert}" ]; then' 2026-01-21 11:38:24.386780 | controller | - ' exit 1' 2026-01-21 11:38:24.386787 | controller | - ' fi' 2026-01-21 11:38:24.386794 | controller | - ' ]' 2026-01-21 11:38:24.386802 | controller | - ' logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386809 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.386827 | controller | - ' logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386835 | controller | | error: cannot exec into a container in a completed pod; current phase is Failed' 2026-01-21 11:38:24.386843 | controller | - ' logger.go:42: 11:23:25 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386850 | controller | | + svc_cert=' 2026-01-21 11:38:24.386858 | controller | - ' logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386865 | controller | | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.386873 | controller | - ' svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat 2026-01-21 11:38:24.386880 | controller | /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.386888 | controller | - ' secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc -o 2026-01-21 11:38:24.386895 | controller | jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.386902 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.386910 | controller | - ' if [ "${svc_cert}" != "${secret_cert}" ]; then' 2026-01-21 11:38:24.386917 | controller | - ' exit 1' 2026-01-21 11:38:24.386925 | controller | - ' fi' 2026-01-21 11:38:24.386932 | controller | - ' ]' 2026-01-21 11:38:24.386939 | controller | - ' logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386947 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.386954 | controller | - ' logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386962 | controller | | error: unable to upgrade connection: container not found ("watcher-api")' 2026-01-21 11:38:24.386970 | controller | - ' logger.go:42: 11:23:26 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.386988 | controller | | + svc_cert=' 2026-01-21 11:38:24.386996 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387004 | controller | | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.387011 | controller | - ' svc_cert=$(oc rsh -n $NAMESPACE -c watcher-api watcher-kuttl-api-0 cat 2026-01-21 11:38:24.387019 | controller | /etc/pki/tls/certs/internal.crt)' 2026-01-21 11:38:24.387026 | controller | - ' secret_cert=$(oc get -n $NAMESPACE secret cert-watcher-internal-svc -o 2026-01-21 11:38:24.387033 | controller | jsonpath=''{.data.tls\.crt}'' | base64 --decode)' 2026-01-21 11:38:24.387041 | controller | - ' # ensure that the svc secret and cert secret match' 2026-01-21 11:38:24.387048 | controller | - ' if [ "${svc_cert}" != "${secret_cert}" ]; then' 2026-01-21 11:38:24.387056 | controller | - ' exit 1' 2026-01-21 11:38:24.387063 | controller | - ' fi' 2026-01-21 11:38:24.387071 | controller | - ' ]' 2026-01-21 11:38:24.387078 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387086 | controller | | ++ oc rsh -n watcher-kuttl-default -c watcher-api watcher-kuttl-api-0 cat /etc/pki/tls/certs/internal.crt' 2026-01-21 11:38:24.387093 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387101 | controller | | + svc_cert=''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.387108 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387116 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.387123 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387130 | controller | | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw' 2026-01-21 11:38:24.387138 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387150 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF' 2026-01-21 11:38:24.387162 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387169 | controller | | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z' 2026-01-21 11:38:24.387177 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387186 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn' 2026-01-21 11:38:24.387197 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387205 | controller | | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P' 2026-01-21 11:38:24.387212 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387220 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58' 2026-01-21 11:38:24.387227 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387235 | controller | | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV' 2026-01-21 11:38:24.387242 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387249 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.387257 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387264 | controller | | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.387271 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387279 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.387286 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387293 | controller | | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy' 2026-01-21 11:38:24.387300 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387308 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD' 2026-01-21 11:38:24.387315 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387322 | controller | | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34' 2026-01-21 11:38:24.387330 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387338 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w' 2026-01-21 11:38:24.387345 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387352 | controller | | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68' 2026-01-21 11:38:24.387360 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387367 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K' 2026-01-21 11:38:24.387374 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387381 | controller | | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG' 2026-01-21 11:38:24.387400 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387409 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2' 2026-01-21 11:38:24.387416 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387423 | controller | | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF' 2026-01-21 11:38:24.387431 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387438 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY' 2026-01-21 11:38:24.387452 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387460 | controller | | P464iw==' 2026-01-21 11:38:24.387468 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387475 | controller | | -----END CERTIFICATE-----''' 2026-01-21 11:38:24.387483 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387490 | controller | | ++ base64 --decode' 2026-01-21 11:38:24.387498 | controller | - ' logger.go:42: 11:23:27 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387505 | controller | | ++ oc get -n watcher-kuttl-default secret cert-watcher-internal-svc -o ''jsonpath={.data.tls\.crt}''' 2026-01-21 11:38:24.387513 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387520 | controller | | + secret_cert=''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.387528 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387535 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.387542 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387550 | controller | | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw' 2026-01-21 11:38:24.387558 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387565 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF' 2026-01-21 11:38:24.387573 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387580 | controller | | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z' 2026-01-21 11:38:24.387588 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387595 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn' 2026-01-21 11:38:24.387602 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387610 | controller | | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P' 2026-01-21 11:38:24.387617 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387625 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58' 2026-01-21 11:38:24.387632 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387649 | controller | | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV' 2026-01-21 11:38:24.387656 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387664 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.387671 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387678 | controller | | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.387686 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387693 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.387700 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387707 | controller | | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy' 2026-01-21 11:38:24.387731 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387743 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD' 2026-01-21 11:38:24.387751 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387763 | controller | | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34' 2026-01-21 11:38:24.387771 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387779 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w' 2026-01-21 11:38:24.387786 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387793 | controller | | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68' 2026-01-21 11:38:24.387801 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387808 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K' 2026-01-21 11:38:24.387826 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387834 | controller | | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG' 2026-01-21 11:38:24.387841 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387849 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2' 2026-01-21 11:38:24.387856 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387864 | controller | | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF' 2026-01-21 11:38:24.387876 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387884 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY' 2026-01-21 11:38:24.387891 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387899 | controller | | P464iw==' 2026-01-21 11:38:24.387906 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387914 | controller | | -----END CERTIFICATE-----''' 2026-01-21 11:38:24.387921 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387928 | controller | | + ''['' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.387936 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387943 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.387951 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387958 | controller | | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw' 2026-01-21 11:38:24.387966 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387973 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF' 2026-01-21 11:38:24.387980 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.387988 | controller | | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z' 2026-01-21 11:38:24.387995 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388003 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn' 2026-01-21 11:38:24.388010 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388018 | controller | | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P' 2026-01-21 11:38:24.388026 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388033 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58' 2026-01-21 11:38:24.388040 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388048 | controller | | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV' 2026-01-21 11:38:24.388059 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388067 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.388074 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388081 | controller | | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.388089 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388096 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.388103 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388110 | controller | | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy' 2026-01-21 11:38:24.388118 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388125 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD' 2026-01-21 11:38:24.388132 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388140 | controller | | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34' 2026-01-21 11:38:24.388147 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388159 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w' 2026-01-21 11:38:24.388167 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388174 | controller | | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68' 2026-01-21 11:38:24.388181 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388188 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K' 2026-01-21 11:38:24.388196 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388203 | controller | | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG' 2026-01-21 11:38:24.388219 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388227 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2' 2026-01-21 11:38:24.388236 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388245 | controller | | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF' 2026-01-21 11:38:24.388253 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388261 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY' 2026-01-21 11:38:24.388270 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388278 | controller | | P464iw==' 2026-01-21 11:38:24.388287 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388295 | controller | | -----END CERTIFICATE-----'' ''!='' ''-----BEGIN CERTIFICATE-----' 2026-01-21 11:38:24.388302 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388309 | controller | | MIID8DCCAligAwIBAgIQfd0EUjS3Upq8kDRSZB2sqTANBgkqhkiG9w0BAQwFADAa' 2026-01-21 11:38:24.388317 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388324 | controller | | MRgwFgYDVQQDEw9yb290Y2EtaW50ZXJuYWwwHhcNMjYwMTIxMTEyMzIzWhcNMzEw' 2026-01-21 11:38:24.388332 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388339 | controller | | MTIwMTEyMzIzWjAAMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtLRF' 2026-01-21 11:38:24.388346 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388358 | controller | | e+OTGNU+uDP/v/uLx/rDaEQbBnBCIKjG+Ftp0yIHgndcK2K+1/Vo3SPzCWQP1z2Z' 2026-01-21 11:38:24.388366 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388374 | controller | | NG4pDVJR9i5bbEICXiI4Yrdx7ZSI+hr7ugjkON+ZzEMsMtYpF9jdkRvmdq5AG1cn' 2026-01-21 11:38:24.388381 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388389 | controller | | RtpPwjVQckwHRn7BAnvIKTHDgzJCQ0YculgISDr51pMkc42imE0Un8PSdq/wGb8P' 2026-01-21 11:38:24.388396 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388403 | controller | | kbz0lqJ/sjdPKIYivlQjGo1KWYFxEjg02VkRC60hEDOwbvJnneEw/oOWr2SyRm58' 2026-01-21 11:38:24.388411 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388418 | controller | | NZ5W+AFPO3J3rxGG6cZFowKRDFtv6+r6K/I2arSOc9Lz4FohEIikFEosrC7OPvOV' 2026-01-21 11:38:24.388426 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388433 | controller | | DFTVbZFIna0hWOz55QIDAQABo4HLMIHIMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUE' 2026-01-21 11:38:24.388440 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388447 | controller | | DDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFKS7aCFOFZkG' 2026-01-21 11:38:24.388455 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388462 | controller | | rbDBcPZ3k3kmpm+iMHIGA1UdEQEB/wRoMGaCKndhdGNoZXItaW50ZXJuYWwud2F0' 2026-01-21 11:38:24.388470 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388477 | controller | | Y2hlci1rdXR0bC1kZWZhdWx0LnN2Y4I4d2F0Y2hlci1pbnRlcm5hbC53YXRjaGVy' 2026-01-21 11:38:24.388484 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388491 | controller | | LWt1dHRsLWRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwwDQYJKoZIhvcNAQEMBQAD' 2026-01-21 11:38:24.388504 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388512 | controller | | ggGBADXM2HesaDU61Y3uVmpS5VTmqzv3FZHXuJuIlBpxKOEVnUt/n9/Kduzd6l34' 2026-01-21 11:38:24.388519 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388526 | controller | | /sBFzH27O4QR4uFtQlNO6HHpe+/a/4uFF41D+7DZjuOqSgQoGG1+rA0zBHSSIe7w' 2026-01-21 11:38:24.388533 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388541 | controller | | SSO2ID9EOKdfsB80pHOJ2gyBIc/zNUVWrX58JDjyr5PqhpY2x9775xJsG7h9ku68' 2026-01-21 11:38:24.388548 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388556 | controller | | UCn4m43SLi/8cAJzo7u5H6vAAtDPHCUm5kP5J7Trtw8dyenC1vsflNNm/22Mhv8K' 2026-01-21 11:38:24.388564 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388571 | controller | | tybyEd5hAy65vdcubBAyKoSBloIH2ReAWyTLUie/sqd9sOOOWImJZu9WcKPn+ZoG' 2026-01-21 11:38:24.388578 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388586 | controller | | OAmRSB1UoP6XC7tmg8JYIr91lQNNpSxFKcvAv94lLne4iDbP3/HfWaiDLhSB18i2' 2026-01-21 11:38:24.388593 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388600 | controller | | NGexsgJm8NJKLOHWxesmpoFIBCJ8MRVqidzWNZdZg5TC6cM7LJhofVUfmA5iP2AF' 2026-01-21 11:38:24.388617 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388626 | controller | | 1tLEBZ+XRVmwovEVTreg+nIQ2Z7MVT4f61lvoovVN2EZFE8sogC7WgdnC3yn5WFY' 2026-01-21 11:38:24.388633 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388645 | controller | | P464iw==' 2026-01-21 11:38:24.388653 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388664 | controller | | -----END CERTIFICATE-----'' '']''' 2026-01-21 11:38:24.388672 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/3-change-internal-svc-certificate 2026-01-21 11:38:24.388679 | controller | | test step completed 3-change-internal-svc-certificate' 2026-01-21 11:38:24.388687 | controller | - ' logger.go:42: 11:23:28 | watcher-tls-certs-change/4-cleanup-watcher | starting 2026-01-21 11:38:24.388694 | controller | test step 4-cleanup-watcher' 2026-01-21 11:38:24.388701 | controller | - ' logger.go:42: 11:23:38 | watcher-tls-certs-change/4-cleanup-watcher | test 2026-01-21 11:38:24.388709 | controller | step completed 4-cleanup-watcher' 2026-01-21 11:38:24.388742 | controller | - ' logger.go:42: 11:23:38 | watcher-tls-certs-change/5-clenaup-certs | starting 2026-01-21 11:38:24.388752 | controller | test step 5-clenaup-certs' 2026-01-21 11:38:24.388760 | controller | - ' logger.go:42: 11:23:38 | watcher-tls-certs-change/5-clenaup-certs | test step 2026-01-21 11:38:24.388767 | controller | completed 5-clenaup-certs' 2026-01-21 11:38:24.388775 | controller | - ' logger.go:42: 11:23:38 | watcher-tls-certs-change | skipping kubernetes event 2026-01-21 11:38:24.388782 | controller | logging' 2026-01-21 11:38:24.388789 | controller | - === CONT kuttl/harness/watcher-tls 2026-01-21 11:38:24.388796 | controller | - ' logger.go:42: 11:23:38 | watcher-tls | Skipping creation of user-supplied namespace: 2026-01-21 11:38:24.388803 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.388811 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/0-cleanup-watcher | starting test step 2026-01-21 11:38:24.388819 | controller | 0-cleanup-watcher' 2026-01-21 11:38:24.388832 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/0-cleanup-watcher | test step completed 2026-01-21 11:38:24.388840 | controller | 0-cleanup-watcher' 2026-01-21 11:38:24.388847 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | starting test step 2026-01-21 11:38:24.388855 | controller | 1-deploy-with-tlse' 2026-01-21 11:38:24.388862 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | Certificate:watcher-kuttl-default/watcher-internal-svc 2026-01-21 11:38:24.388870 | controller | created' 2026-01-21 11:38:24.388877 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | Certificate:watcher-kuttl-default/watcher-public-svc 2026-01-21 11:38:24.388886 | controller | created' 2026-01-21 11:38:24.388895 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.388904 | controller | created' 2026-01-21 11:38:24.388913 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.388922 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.388931 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.388940 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.388948 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.388957 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.388966 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.388975 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.388983 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.388992 | controller | ]' 2026-01-21 11:38:24.389001 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.my\.cnf}''|base64 2026-01-21 11:38:24.389009 | controller | -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.389018 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.00-default\.conf}''|base64 2026-01-21 11:38:24.389032 | controller | -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" == 2 2026-01-21 11:38:24.389041 | controller | ]' 2026-01-21 11:38:24.389050 | controller | - ' # check that both endpoints have https set' 2026-01-21 11:38:24.389057 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.389065 | controller | infra-optim | [ $(grep -c https) == 2 ]' 2026-01-21 11:38:24.389073 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.389081 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.389089 | controller | == "" ]; then' 2026-01-21 11:38:24.389096 | controller | - ' exit 0' 2026-01-21 11:38:24.389102 | controller | - ' fi' 2026-01-21 11:38:24.389107 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.389113 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.389119 | controller | - ' counter=0' 2026-01-21 11:38:24.389125 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.389131 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.389137 | controller | - ' echo ${i}' 2026-01-21 11:38:24.389143 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.389149 | controller | - ' fi' 2026-01-21 11:38:24.389155 | controller | - ' done' 2026-01-21 11:38:24.389161 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.389178 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.389185 | controller | - ' exit 1' 2026-01-21 11:38:24.389191 | controller | - ' else' 2026-01-21 11:38:24.389197 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.389203 | controller | - ' fi' 2026-01-21 11:38:24.389209 | controller | - ' ]' 2026-01-21 11:38:24.389215 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.389221 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.389227 | controller | - ' logger.go:42: 11:23:38 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.389235 | controller | - ' logger.go:42: 11:23:44 | watcher-tls/1-deploy-with-tlse | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.389242 | controller | - ' logger.go:42: 11:23:45 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.389249 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.389256 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.389262 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.389268 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.389274 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.389280 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.389286 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.389292 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.389298 | controller | ]' 2026-01-21 11:38:24.389309 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.my\.cnf}''|base64 2026-01-21 11:38:24.389316 | controller | -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.389322 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.00-default\.conf}''|base64 2026-01-21 11:38:24.389328 | controller | -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" == 2 2026-01-21 11:38:24.389334 | controller | ]' 2026-01-21 11:38:24.389340 | controller | - ' # check that both endpoints have https set' 2026-01-21 11:38:24.389346 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.389356 | controller | infra-optim | [ $(grep -c https) == 2 ]' 2026-01-21 11:38:24.389362 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.389368 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.389374 | controller | == "" ]; then' 2026-01-21 11:38:24.389380 | controller | - ' exit 0' 2026-01-21 11:38:24.389386 | controller | - ' fi' 2026-01-21 11:38:24.389392 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.389398 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.389404 | controller | - ' counter=0' 2026-01-21 11:38:24.389410 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.389416 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.389422 | controller | - ' echo ${i}' 2026-01-21 11:38:24.389428 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.389434 | controller | - ' fi' 2026-01-21 11:38:24.389440 | controller | - ' done' 2026-01-21 11:38:24.389446 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.389452 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.389458 | controller | - ' exit 1' 2026-01-21 11:38:24.389464 | controller | - ' else' 2026-01-21 11:38:24.389470 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.389476 | controller | - ' fi' 2026-01-21 11:38:24.389482 | controller | - ' ]' 2026-01-21 11:38:24.389488 | controller | - ' logger.go:42: 11:23:45 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.389498 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.389504 | controller | - ' logger.go:42: 11:23:45 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.389509 | controller | - ' logger.go:42: 11:23:48 | watcher-tls/1-deploy-with-tlse | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.389515 | controller | - ' logger.go:42: 11:23:49 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.389521 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.389527 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.389533 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.389539 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.389547 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.389554 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.389562 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.389570 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.389577 | controller | ]' 2026-01-21 11:38:24.389586 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.my\.cnf}''|base64 2026-01-21 11:38:24.389593 | controller | -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.389601 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.00-default\.conf}''|base64 2026-01-21 11:38:24.389619 | controller | -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" == 2 2026-01-21 11:38:24.389627 | controller | ]' 2026-01-21 11:38:24.389635 | controller | - ' # check that both endpoints have https set' 2026-01-21 11:38:24.389643 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.389650 | controller | infra-optim | [ $(grep -c https) == 2 ]' 2026-01-21 11:38:24.389658 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.389670 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.389677 | controller | == "" ]; then' 2026-01-21 11:38:24.389683 | controller | - ' exit 0' 2026-01-21 11:38:24.389689 | controller | - ' fi' 2026-01-21 11:38:24.389695 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.389705 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.389711 | controller | - ' counter=0' 2026-01-21 11:38:24.389735 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.389741 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.389747 | controller | - ' echo ${i}' 2026-01-21 11:38:24.389753 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.389759 | controller | - ' fi' 2026-01-21 11:38:24.389765 | controller | - ' done' 2026-01-21 11:38:24.389771 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.389777 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.389783 | controller | - ' exit 1' 2026-01-21 11:38:24.389789 | controller | - ' else' 2026-01-21 11:38:24.389796 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.389804 | controller | - ' fi' 2026-01-21 11:38:24.389811 | controller | - ' ]' 2026-01-21 11:38:24.389819 | controller | - ' logger.go:42: 11:23:49 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.389826 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.389834 | controller | - ' logger.go:42: 11:23:49 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.389841 | controller | - ' logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.389847 | controller | - ' logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.389853 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.389859 | controller | - ' logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse | ++ awk ''{print $1}''' 2026-01-21 11:38:24.389866 | controller | - ' logger.go:42: 11:23:53 | watcher-tls/1-deploy-with-tlse | ++ grep watcher' 2026-01-21 11:38:24.389874 | controller | - ' logger.go:42: 11:23:57 | watcher-tls/1-deploy-with-tlse | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.389882 | controller | - ' logger.go:42: 11:23:57 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.389890 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.389897 | controller | - ' logger.go:42: 11:23:58 | watcher-tls/1-deploy-with-tlse | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.389905 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.389913 | controller | - ' logger.go:42: 11:23:58 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.389920 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.389928 | controller | - ' logger.go:42: 11:23:58 | watcher-tls/1-deploy-with-tlse | + ''['' -n '''' '']''' 2026-01-21 11:38:24.389936 | controller | - ' logger.go:42: 11:23:59 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.389943 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.389951 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.389957 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.389963 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.389969 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.389975 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.389989 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.389995 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.390001 | controller | ]' 2026-01-21 11:38:24.390007 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.my\.cnf}''|base64 2026-01-21 11:38:24.390013 | controller | -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.390019 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.00-default\.conf}''|base64 2026-01-21 11:38:24.390025 | controller | -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" == 2 2026-01-21 11:38:24.390032 | controller | ]' 2026-01-21 11:38:24.390040 | controller | - ' # check that both endpoints have https set' 2026-01-21 11:38:24.390050 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.390056 | controller | infra-optim | [ $(grep -c https) == 2 ]' 2026-01-21 11:38:24.390062 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.390079 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.390085 | controller | == "" ]; then' 2026-01-21 11:38:24.390091 | controller | - ' exit 0' 2026-01-21 11:38:24.390097 | controller | - ' fi' 2026-01-21 11:38:24.390103 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.390109 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.390115 | controller | - ' counter=0' 2026-01-21 11:38:24.390122 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.390130 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.390138 | controller | - ' echo ${i}' 2026-01-21 11:38:24.390145 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.390152 | controller | - ' fi' 2026-01-21 11:38:24.390160 | controller | - ' done' 2026-01-21 11:38:24.390168 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.390175 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.390190 | controller | - ' exit 1' 2026-01-21 11:38:24.390198 | controller | - ' else' 2026-01-21 11:38:24.390205 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.390212 | controller | - ' fi' 2026-01-21 11:38:24.390218 | controller | - ' ]' 2026-01-21 11:38:24.390224 | controller | - ' logger.go:42: 11:23:59 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.390230 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.390236 | controller | - ' logger.go:42: 11:23:59 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.390242 | controller | - ' logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.390248 | controller | - ' logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.390254 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.390259 | controller | - ' logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse | ++ awk ''{print $1}''' 2026-01-21 11:38:24.390265 | controller | - ' logger.go:42: 11:24:03 | watcher-tls/1-deploy-with-tlse | ++ grep watcher' 2026-01-21 11:38:24.390271 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.390277 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390283 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.390289 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.390299 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.390305 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390311 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.390318 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.390325 | controller | '']''' 2026-01-21 11:38:24.390331 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390337 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.my\.cnf}''' 2026-01-21 11:38:24.390343 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ grep -c ssl=1' 2026-01-21 11:38:24.390349 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ base64 -d' 2026-01-21 11:38:24.390355 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.390360 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390366 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.00-default\.conf}''' 2026-01-21 11:38:24.390372 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''cafile 2026-01-21 11:38:24.390378 | controller | = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem''' 2026-01-21 11:38:24.390384 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ base64 -d' 2026-01-21 11:38:24.390390 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.390396 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.390402 | controller | openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.390408 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | + grep infra-optim' 2026-01-21 11:38:24.390413 | controller | - ' logger.go:42: 11:24:06 | watcher-tls/1-deploy-with-tlse | ++ grep -c https' 2026-01-21 11:38:24.390419 | controller | - ' logger.go:42: 11:24:09 | watcher-tls/1-deploy-with-tlse | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.390425 | controller | - ' logger.go:42: 11:24:10 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.390431 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.390437 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.390443 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.390449 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.390470 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.390477 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.390483 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.390489 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.390496 | controller | ]' 2026-01-21 11:38:24.390502 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.my\.cnf}''|base64 2026-01-21 11:38:24.390508 | controller | -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.390514 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.00-default\.conf}''|base64 2026-01-21 11:38:24.390520 | controller | -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" == 2 2026-01-21 11:38:24.390526 | controller | ]' 2026-01-21 11:38:24.390532 | controller | - ' # check that both endpoints have https set' 2026-01-21 11:38:24.390538 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.390549 | controller | infra-optim | [ $(grep -c https) == 2 ]' 2026-01-21 11:38:24.390562 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.390570 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.390600 | controller | == "" ]; then' 2026-01-21 11:38:24.390608 | controller | - ' exit 0' 2026-01-21 11:38:24.390615 | controller | - ' fi' 2026-01-21 11:38:24.390621 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.390627 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.390633 | controller | - ' counter=0' 2026-01-21 11:38:24.390639 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.390645 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.390651 | controller | - ' echo ${i}' 2026-01-21 11:38:24.390657 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.390664 | controller | - ' fi' 2026-01-21 11:38:24.390669 | controller | - ' done' 2026-01-21 11:38:24.390675 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.390681 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.390687 | controller | - ' exit 1' 2026-01-21 11:38:24.390694 | controller | - ' else' 2026-01-21 11:38:24.390710 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.390747 | controller | - ' fi' 2026-01-21 11:38:24.390756 | controller | - ' ]' 2026-01-21 11:38:24.390764 | controller | - ' logger.go:42: 11:24:10 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.390772 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.390779 | controller | - ' logger.go:42: 11:24:10 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.390787 | controller | - ' logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.390794 | controller | - ' logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.390800 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.390806 | controller | - ' logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse | ++ awk ''{print $1}''' 2026-01-21 11:38:24.390811 | controller | - ' logger.go:42: 11:24:13 | watcher-tls/1-deploy-with-tlse | ++ grep watcher' 2026-01-21 11:38:24.390817 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.390823 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390829 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.390835 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.390841 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.390847 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390853 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.390858 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.390870 | controller | '']''' 2026-01-21 11:38:24.390876 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ base64 -d' 2026-01-21 11:38:24.390882 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ grep -c ssl=1' 2026-01-21 11:38:24.390887 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390899 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.my\.cnf}''' 2026-01-21 11:38:24.390909 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.390915 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.390922 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.00-default\.conf}''' 2026-01-21 11:38:24.390942 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''cafile 2026-01-21 11:38:24.390949 | controller | = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem''' 2026-01-21 11:38:24.390955 | controller | - ' logger.go:42: 11:24:16 | watcher-tls/1-deploy-with-tlse | ++ base64 -d' 2026-01-21 11:38:24.390961 | controller | - ' logger.go:42: 11:24:17 | watcher-tls/1-deploy-with-tlse | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.390967 | controller | - ' logger.go:42: 11:24:17 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.390973 | controller | openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.390979 | controller | - ' logger.go:42: 11:24:17 | watcher-tls/1-deploy-with-tlse | + grep infra-optim' 2026-01-21 11:38:24.390985 | controller | - ' logger.go:42: 11:24:17 | watcher-tls/1-deploy-with-tlse | ++ grep -c https' 2026-01-21 11:38:24.390991 | controller | - ' logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.390997 | controller | - ' logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse | ++ oc get pods -n 2026-01-21 11:38:24.391003 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.391009 | controller | - ' logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse | + ''['' '''' == '''' 2026-01-21 11:38:24.391015 | controller | '']''' 2026-01-21 11:38:24.391021 | controller | - ' logger.go:42: 11:24:19 | watcher-tls/1-deploy-with-tlse | + exit 0' 2026-01-21 11:38:24.391027 | controller | - ' logger.go:42: 11:24:20 | watcher-tls/1-deploy-with-tlse | running command: 2026-01-21 11:38:24.391033 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.391039 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.391044 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.391050 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.391056 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.391062 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.391068 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.391074 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.391079 | controller | ]' 2026-01-21 11:38:24.391086 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.my\.cnf}''|base64 2026-01-21 11:38:24.391091 | controller | -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.391097 | controller | - ' [ "$(oc get -n ${NAMESPACE} secret watcher-kuttl-api-config-data -o jsonpath=''{.data.00-default\.conf}''|base64 2026-01-21 11:38:24.391103 | controller | -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" == 2 2026-01-21 11:38:24.391109 | controller | ]' 2026-01-21 11:38:24.391115 | controller | - ' # check that both endpoints have https set' 2026-01-21 11:38:24.391121 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.391127 | controller | infra-optim | [ $(grep -c https) == 2 ]' 2026-01-21 11:38:24.391133 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.391139 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.391144 | controller | == "" ]; then' 2026-01-21 11:38:24.391150 | controller | - ' exit 0' 2026-01-21 11:38:24.391156 | controller | - ' fi' 2026-01-21 11:38:24.391166 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.391172 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.391178 | controller | - ' counter=0' 2026-01-21 11:38:24.391184 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.391189 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.391195 | controller | - ' echo ${i}' 2026-01-21 11:38:24.391201 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.391207 | controller | - ' fi' 2026-01-21 11:38:24.391217 | controller | - ' done' 2026-01-21 11:38:24.391223 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.391229 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.391236 | controller | - ' exit 1' 2026-01-21 11:38:24.391243 | controller | - ' else' 2026-01-21 11:38:24.391249 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.391255 | controller | - ' fi' 2026-01-21 11:38:24.391262 | controller | - ' ]' 2026-01-21 11:38:24.391269 | controller | - ' logger.go:42: 11:24:20 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.391275 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.391283 | controller | - ' logger.go:42: 11:24:20 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.391290 | controller | - ' logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.391295 | controller | - ' logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse | ++ grep watcher' 2026-01-21 11:38:24.391301 | controller | - ' logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse | ++ awk ''{print $1}''' 2026-01-21 11:38:24.391307 | controller | - ' logger.go:42: 11:24:23 | watcher-tls/1-deploy-with-tlse | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.391313 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.391319 | controller | - ' logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.391338 | controller | - ' logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.391345 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.391351 | controller | - ' logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.391357 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.391363 | controller | - ' logger.go:42: 11:24:25 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.391369 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.391375 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.391381 | controller | '']''' 2026-01-21 11:38:24.391387 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ base64 -d' 2026-01-21 11:38:24.391393 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ grep -c ssl=1' 2026-01-21 11:38:24.391399 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.391405 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.my\.cnf}''' 2026-01-21 11:38:24.391411 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.391417 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.391423 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.00-default\.conf}''' 2026-01-21 11:38:24.391428 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ grep -c ''cafile 2026-01-21 11:38:24.391438 | controller | = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem''' 2026-01-21 11:38:24.391444 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ base64 -d' 2026-01-21 11:38:24.391450 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.391456 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.391462 | controller | openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.391468 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | + grep infra-optim' 2026-01-21 11:38:24.391474 | controller | - ' logger.go:42: 11:24:26 | watcher-tls/1-deploy-with-tlse | ++ grep -c https' 2026-01-21 11:38:24.391480 | controller | - ' logger.go:42: 11:24:28 | watcher-tls/1-deploy-with-tlse | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.391485 | controller | - ' logger.go:42: 11:24:28 | watcher-tls/1-deploy-with-tlse | ++ oc get pods -n 2026-01-21 11:38:24.391491 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.391497 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/1-deploy-with-tlse | + ''['' '''' == '''' 2026-01-21 11:38:24.391503 | controller | '']''' 2026-01-21 11:38:24.391509 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/1-deploy-with-tlse | + exit 0' 2026-01-21 11:38:24.391515 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/1-deploy-with-tlse | test step completed 2026-01-21 11:38:24.391521 | controller | 1-deploy-with-tlse' 2026-01-21 11:38:24.391533 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | starting test step 2-patch-mtls' 2026-01-21 11:38:24.391539 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.391546 | controller | set -euxo pipefail' 2026-01-21 11:38:24.391558 | controller | - ' oc patch oscp -n $NAMESPACE openstack --type=''json'' -p=''[{"op": "replace", 2026-01-21 11:38:24.391573 | controller | "path": "/spec/memcached/templates/memcached/tls/mtls/sslVerifyMode", "value": "Request"}]''' 2026-01-21 11:38:24.391582 | controller | - ' ]' 2026-01-21 11:38:24.391588 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + oc patch oscp -n watcher-kuttl-default 2026-01-21 11:38:24.391594 | controller | openstack --type=json ''-p=[{"op": "replace", "path": "/spec/memcached/templates/memcached/tls/mtls/sslVerifyMode", 2026-01-21 11:38:24.391600 | controller | "value": "Request"}]''' 2026-01-21 11:38:24.391606 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | openstackcontrolplane.core.openstack.org/openstack 2026-01-21 11:38:24.391612 | controller | patched' 2026-01-21 11:38:24.391618 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.391624 | controller | set -euxo pipefail' 2026-01-21 11:38:24.391630 | controller | - ' ' 2026-01-21 11:38:24.391637 | controller | - ' oc project ${NAMESPACE}' 2026-01-21 11:38:24.391643 | controller | - ' # Get pod names for each watcher service' 2026-01-21 11:38:24.391649 | controller | - ' APIPOD=$(oc get pods -l service=watcher-api -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.391655 | controller | - ' APPLIERPOD=$(oc get pods -l service=watcher-applier -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.391661 | controller | - ' DECISIONENGINEPOD=$(oc get pods -l service=watcher-decision-engine -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.391667 | controller | - ' ' 2026-01-21 11:38:24.391673 | controller | - ' # Verify memcached mTLS config parameters in watcher-api config' 2026-01-21 11:38:24.391679 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.391685 | controller | - ' echo "Checking watcher-api config..."' 2026-01-21 11:38:24.391704 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391711 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.391750 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391756 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.391762 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391768 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.391774 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391780 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.391786 | controller | - ' ' 2026-01-21 11:38:24.391793 | controller | - ' # Verify mTLS config parameters in memcached backend config' 2026-01-21 11:38:24.391798 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391804 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.391810 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391817 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.391825 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391842 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.391853 | controller | - ' else' 2026-01-21 11:38:24.391861 | controller | - ' exit 1' 2026-01-21 11:38:24.391869 | controller | - ' fi' 2026-01-21 11:38:24.391876 | controller | - ' ' 2026-01-21 11:38:24.391883 | controller | - ' # Verify memcached mTLS config parameters in watcher-applier config' 2026-01-21 11:38:24.391890 | controller | - ' if [ -n "${APPLIERPOD}" ]; then' 2026-01-21 11:38:24.391898 | controller | - ' echo "Checking watcher-applier config..."' 2026-01-21 11:38:24.391906 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391914 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.391921 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391929 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.391936 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391944 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.391952 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391959 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.391965 | controller | - ' ' 2026-01-21 11:38:24.391971 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391977 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.391983 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.391989 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.391995 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392007 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392036 | controller | - ' else' 2026-01-21 11:38:24.392043 | controller | - ' exit 1' 2026-01-21 11:38:24.392049 | controller | - ' fi' 2026-01-21 11:38:24.392055 | controller | - ' ' 2026-01-21 11:38:24.392061 | controller | - ' # Verify memcached mTLS config parameters in watcher-decision-engine config' 2026-01-21 11:38:24.392072 | controller | - ' if [ -n "${DECISIONENGINEPOD}" ]; then' 2026-01-21 11:38:24.392078 | controller | - ' echo "Checking watcher-decision-engine config..."' 2026-01-21 11:38:24.392084 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392090 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.392096 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392105 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392111 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392117 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392123 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392129 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.392135 | controller | - ' ' 2026-01-21 11:38:24.392158 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392166 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.392173 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392180 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392187 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392193 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392199 | controller | - ' else' 2026-01-21 11:38:24.392204 | controller | - ' exit 1' 2026-01-21 11:38:24.392210 | controller | - ' fi' 2026-01-21 11:38:24.392216 | controller | - ' ]' 2026-01-21 11:38:24.392222 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.392228 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | Now using project "watcher-kuttl-default" 2026-01-21 11:38:24.392234 | controller | on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.392240 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.392250 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.392255 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.392261 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-applier 2026-01-21 11:38:24.392267 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.392273 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0' 2026-01-21 11:38:24.392279 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.392285 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.392291 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.392297 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-api-0 2026-01-21 11:38:24.392303 | controller | '']''' 2026-01-21 11:38:24.392309 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-api 2026-01-21 11:38:24.392314 | controller | config...''' 2026-01-21 11:38:24.392320 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | Checking watcher-api config...' 2026-01-21 11:38:24.392333 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.392339 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.392345 | controller | - ' logger.go:42: 11:24:29 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.392351 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.392357 | controller | - ' logger.go:42: 11:24:30 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.392363 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.392369 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.392375 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.392382 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.392388 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.392394 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.392400 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.392406 | controller | project_domain_name = Default project_name = service user_domain_name = Default 2026-01-21 11:38:24.392412 | controller | password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.392418 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.392423 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.392429 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.392436 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.392442 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.392457 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.392463 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.392469 | controller | tls_enabled=true memcache_dead_retry = 30 ''[prometheus_client]'' host = metric-storage-prometheus.watcher-kuttl-default.svc 2026-01-21 11:38:24.392475 | controller | port = 9090 cafile = /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' 2026-01-21 11:38:24.392481 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.392487 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.392493 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.392499 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.392505 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.392511 | controller | period = 900' 2026-01-21 11:38:24.392516 | controller | - ' logger.go:42: 11:24:30 | watcher-tls/2-patch-mtls | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.392522 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.392528 | controller | set -euxo pipefail' 2026-01-21 11:38:24.392540 | controller | - ' ' 2026-01-21 11:38:24.392547 | controller | - ' oc project ${NAMESPACE}' 2026-01-21 11:38:24.392565 | controller | - ' # Get pod names for each watcher service' 2026-01-21 11:38:24.392574 | controller | - ' APIPOD=$(oc get pods -l service=watcher-api -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.392581 | controller | - ' APPLIERPOD=$(oc get pods -l service=watcher-applier -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.392589 | controller | - ' DECISIONENGINEPOD=$(oc get pods -l service=watcher-decision-engine -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.392596 | controller | - ' ' 2026-01-21 11:38:24.392604 | controller | - ' # Verify memcached mTLS config parameters in watcher-api config' 2026-01-21 11:38:24.392612 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.392620 | controller | - ' echo "Checking watcher-api config..."' 2026-01-21 11:38:24.392627 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392635 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.392642 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392650 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392657 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392665 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392673 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392680 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.392688 | controller | - ' ' 2026-01-21 11:38:24.392695 | controller | - ' # Verify mTLS config parameters in memcached backend config' 2026-01-21 11:38:24.392702 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392709 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.392744 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392752 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392758 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392764 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392769 | controller | - ' else' 2026-01-21 11:38:24.392775 | controller | - ' exit 1' 2026-01-21 11:38:24.392781 | controller | - ' fi' 2026-01-21 11:38:24.392787 | controller | - ' ' 2026-01-21 11:38:24.392793 | controller | - ' # Verify memcached mTLS config parameters in watcher-applier config' 2026-01-21 11:38:24.392799 | controller | - ' if [ -n "${APPLIERPOD}" ]; then' 2026-01-21 11:38:24.392805 | controller | - ' echo "Checking watcher-applier config..."' 2026-01-21 11:38:24.392810 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392816 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.392822 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392828 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392834 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392840 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392845 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392868 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.392875 | controller | - ' ' 2026-01-21 11:38:24.392881 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392887 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.392893 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392899 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392905 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392911 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392917 | controller | - ' else' 2026-01-21 11:38:24.392923 | controller | - ' exit 1' 2026-01-21 11:38:24.392934 | controller | - ' fi' 2026-01-21 11:38:24.392940 | controller | - ' ' 2026-01-21 11:38:24.392946 | controller | - ' # Verify memcached mTLS config parameters in watcher-decision-engine config' 2026-01-21 11:38:24.392952 | controller | - ' if [ -n "${DECISIONENGINEPOD}" ]; then' 2026-01-21 11:38:24.392958 | controller | - ' echo "Checking watcher-decision-engine config..."' 2026-01-21 11:38:24.392964 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392970 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.392976 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392981 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.392987 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.392993 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.392999 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393005 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.393011 | controller | - ' ' 2026-01-21 11:38:24.393017 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393023 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.393028 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393034 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393040 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393046 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393052 | controller | - ' else' 2026-01-21 11:38:24.393058 | controller | - ' exit 1' 2026-01-21 11:38:24.393064 | controller | - ' fi' 2026-01-21 11:38:24.393069 | controller | - ' ]' 2026-01-21 11:38:24.393075 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.393081 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | Already on project "watcher-kuttl-default" 2026-01-21 11:38:24.393087 | controller | on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.393093 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.393099 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.393105 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.393131 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-applier 2026-01-21 11:38:24.393138 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.393144 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0' 2026-01-21 11:38:24.393149 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.393155 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.393161 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.393167 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-api-0 2026-01-21 11:38:24.393173 | controller | '']''' 2026-01-21 11:38:24.393179 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-api 2026-01-21 11:38:24.393185 | controller | config...''' 2026-01-21 11:38:24.393191 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | Checking watcher-api config...' 2026-01-21 11:38:24.393205 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.393212 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.393218 | controller | - ' logger.go:42: 11:24:31 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.393224 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.393240 | controller | - ' logger.go:42: 11:24:32 | watcher-tls/2-patch-mtls | error: Internal error occurred: 2026-01-21 11:38:24.393247 | controller | error executing command in container: container is not created or running' 2026-01-21 11:38:24.393253 | controller | - ' logger.go:42: 11:24:32 | watcher-tls/2-patch-mtls | ++ echo' 2026-01-21 11:38:24.393259 | controller | - ' logger.go:42: 11:24:32 | watcher-tls/2-patch-mtls | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.393266 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.393273 | controller | set -euxo pipefail' 2026-01-21 11:38:24.393280 | controller | - ' ' 2026-01-21 11:38:24.393287 | controller | - ' oc project ${NAMESPACE}' 2026-01-21 11:38:24.393294 | controller | - ' # Get pod names for each watcher service' 2026-01-21 11:38:24.393303 | controller | - ' APIPOD=$(oc get pods -l service=watcher-api -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.393309 | controller | - ' APPLIERPOD=$(oc get pods -l service=watcher-applier -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.393315 | controller | - ' DECISIONENGINEPOD=$(oc get pods -l service=watcher-decision-engine -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.393321 | controller | - ' ' 2026-01-21 11:38:24.393326 | controller | - ' # Verify memcached mTLS config parameters in watcher-api config' 2026-01-21 11:38:24.393332 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.393341 | controller | - ' echo "Checking watcher-api config..."' 2026-01-21 11:38:24.393347 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393353 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.393359 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393365 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393371 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393377 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393383 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393389 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.393401 | controller | - ' ' 2026-01-21 11:38:24.393407 | controller | - ' # Verify mTLS config parameters in memcached backend config' 2026-01-21 11:38:24.393413 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393419 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.393425 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393431 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393437 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393443 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393448 | controller | - ' else' 2026-01-21 11:38:24.393454 | controller | - ' exit 1' 2026-01-21 11:38:24.393460 | controller | - ' fi' 2026-01-21 11:38:24.393466 | controller | - ' ' 2026-01-21 11:38:24.393472 | controller | - ' # Verify memcached mTLS config parameters in watcher-applier config' 2026-01-21 11:38:24.393478 | controller | - ' if [ -n "${APPLIERPOD}" ]; then' 2026-01-21 11:38:24.393484 | controller | - ' echo "Checking watcher-applier config..."' 2026-01-21 11:38:24.393490 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393496 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.393502 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393508 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393513 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393519 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393525 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393531 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.393537 | controller | - ' ' 2026-01-21 11:38:24.393543 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393549 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.393555 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393561 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393567 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393572 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393578 | controller | - ' else' 2026-01-21 11:38:24.393584 | controller | - ' exit 1' 2026-01-21 11:38:24.393590 | controller | - ' fi' 2026-01-21 11:38:24.393596 | controller | - ' ' 2026-01-21 11:38:24.393611 | controller | - ' # Verify memcached mTLS config parameters in watcher-decision-engine config' 2026-01-21 11:38:24.393618 | controller | - ' if [ -n "${DECISIONENGINEPOD}" ]; then' 2026-01-21 11:38:24.393624 | controller | - ' echo "Checking watcher-decision-engine config..."' 2026-01-21 11:38:24.393630 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393636 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.393642 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393648 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393657 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393663 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393695 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393701 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.393707 | controller | - ' ' 2026-01-21 11:38:24.393733 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393739 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.393745 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393751 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.393757 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.393766 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.393772 | controller | - ' else' 2026-01-21 11:38:24.393778 | controller | - ' exit 1' 2026-01-21 11:38:24.393784 | controller | - ' fi' 2026-01-21 11:38:24.393790 | controller | - ' ]' 2026-01-21 11:38:24.393796 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.393802 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | Already on project "watcher-kuttl-default" 2026-01-21 11:38:24.393808 | controller | on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.393814 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.393820 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.393826 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.393832 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-applier 2026-01-21 11:38:24.393837 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.393843 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0' 2026-01-21 11:38:24.393849 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.393855 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.393861 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.393867 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-api-0 2026-01-21 11:38:24.393873 | controller | '']''' 2026-01-21 11:38:24.393879 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-api 2026-01-21 11:38:24.393885 | controller | config...''' 2026-01-21 11:38:24.393891 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | Checking watcher-api config...' 2026-01-21 11:38:24.393897 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.393903 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.393909 | controller | - ' logger.go:42: 11:24:33 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.393915 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.393920 | controller | - ' logger.go:42: 11:24:34 | watcher-tls/2-patch-mtls | error: unable to upgrade 2026-01-21 11:38:24.393926 | controller | connection: container not found ("watcher-api")' 2026-01-21 11:38:24.393932 | controller | - ' logger.go:42: 11:24:34 | watcher-tls/2-patch-mtls | ++ echo' 2026-01-21 11:38:24.393952 | controller | - ' logger.go:42: 11:24:34 | watcher-tls/2-patch-mtls | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.393958 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.393964 | controller | set -euxo pipefail' 2026-01-21 11:38:24.393970 | controller | - ' ' 2026-01-21 11:38:24.393976 | controller | - ' oc project ${NAMESPACE}' 2026-01-21 11:38:24.393982 | controller | - ' # Get pod names for each watcher service' 2026-01-21 11:38:24.393988 | controller | - ' APIPOD=$(oc get pods -l service=watcher-api -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.393994 | controller | - ' APPLIERPOD=$(oc get pods -l service=watcher-applier -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.394010 | controller | - ' DECISIONENGINEPOD=$(oc get pods -l service=watcher-decision-engine -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.394024 | controller | - ' ' 2026-01-21 11:38:24.394039 | controller | - ' # Verify memcached mTLS config parameters in watcher-api config' 2026-01-21 11:38:24.394048 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.394056 | controller | - ' echo "Checking watcher-api config..."' 2026-01-21 11:38:24.394064 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394072 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.394079 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394087 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.394095 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394103 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.394111 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394118 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.394126 | controller | - ' ' 2026-01-21 11:38:24.394133 | controller | - ' # Verify mTLS config parameters in memcached backend config' 2026-01-21 11:38:24.394140 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394147 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.394154 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394162 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.394169 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394174 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.394180 | controller | - ' else' 2026-01-21 11:38:24.394186 | controller | - ' exit 1' 2026-01-21 11:38:24.394192 | controller | - ' fi' 2026-01-21 11:38:24.394198 | controller | - ' ' 2026-01-21 11:38:24.394204 | controller | - ' # Verify memcached mTLS config parameters in watcher-applier config' 2026-01-21 11:38:24.394216 | controller | - ' if [ -n "${APPLIERPOD}" ]; then' 2026-01-21 11:38:24.394222 | controller | - ' echo "Checking watcher-applier config..."' 2026-01-21 11:38:24.394228 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394234 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.394240 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394246 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.394268 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394275 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.394281 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394287 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.394293 | controller | - ' ' 2026-01-21 11:38:24.394299 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394314 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.394320 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394326 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.394332 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394338 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.394344 | controller | - ' else' 2026-01-21 11:38:24.394350 | controller | - ' exit 1' 2026-01-21 11:38:24.394356 | controller | - ' fi' 2026-01-21 11:38:24.394362 | controller | - ' ' 2026-01-21 11:38:24.394368 | controller | - ' # Verify memcached mTLS config parameters in watcher-decision-engine config' 2026-01-21 11:38:24.394374 | controller | - ' if [ -n "${DECISIONENGINEPOD}" ]; then' 2026-01-21 11:38:24.394380 | controller | - ' echo "Checking watcher-decision-engine config..."' 2026-01-21 11:38:24.394386 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394392 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.394397 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394403 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.394409 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394432 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.394439 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394445 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.394452 | controller | - ' ' 2026-01-21 11:38:24.394464 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394479 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.394487 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394494 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.394501 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.394510 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.394517 | controller | - ' else' 2026-01-21 11:38:24.394538 | controller | - ' exit 1' 2026-01-21 11:38:24.394546 | controller | - ' fi' 2026-01-21 11:38:24.394553 | controller | - ' ]' 2026-01-21 11:38:24.394560 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.394567 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | Already on project "watcher-kuttl-default" 2026-01-21 11:38:24.394589 | controller | on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.394661 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.394673 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.394681 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.394701 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-applier 2026-01-21 11:38:24.394728 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.394743 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0' 2026-01-21 11:38:24.394751 | controller | - ' logger.go:42: 11:24:35 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.394759 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.394766 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.394774 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-api-0 2026-01-21 11:38:24.394781 | controller | '']''' 2026-01-21 11:38:24.394788 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-api 2026-01-21 11:38:24.394796 | controller | config...''' 2026-01-21 11:38:24.394803 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | Checking watcher-api config...' 2026-01-21 11:38:24.394811 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.394818 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.394825 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.394833 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.394840 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.394848 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.394855 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.394863 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.394870 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.394878 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.394885 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.394893 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.394900 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.394908 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.394915 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.394922 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.394930 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.394957 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.394968 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.394982 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.394990 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.394997 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395005 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.395012 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.395020 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.395028 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.395035 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.395042 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.395050 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.395058 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.395065 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.395073 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.395080 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.395088 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_keyfile 2026-01-21 11:38:24.395107 | controller | = /etc/pki/tls/private/mtls.key''' 2026-01-21 11:38:24.395117 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.395124 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.395132 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.395139 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.395146 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.395153 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.395161 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.395168 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.395175 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.395183 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395191 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.395199 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.395207 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.395215 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.395222 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.395248 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.395257 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.395264 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.395272 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.395280 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395288 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.395309 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.395318 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.395325 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.395333 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.395340 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.395347 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.395354 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.395362 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.395370 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.395377 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.395384 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_cafile 2026-01-21 11:38:24.395392 | controller | = /etc/pki/tls/certs/mtls-ca.crt''' 2026-01-21 11:38:24.395399 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.395406 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.395414 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.395422 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.395436 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.395444 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.395451 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.395459 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.395466 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.395474 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395481 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.395488 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.395499 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.395526 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.395535 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.395543 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.395550 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.395558 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.395565 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.395573 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395581 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.395589 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.395596 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.395604 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.395611 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.395618 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.395625 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.395632 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.395651 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.395658 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.395663 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.395669 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_enabled 2026-01-21 11:38:24.395675 | controller | = true''' 2026-01-21 11:38:24.395682 | controller | - ' logger.go:42: 11:24:36 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.395688 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.395696 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.395735 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.395751 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.395759 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.395767 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.395774 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.395786 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.395802 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395810 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.395834 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.395842 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.395848 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.395854 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.395860 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.395866 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.395874 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.395898 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.395907 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.395914 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.395922 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.395930 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.395937 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.395945 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.395953 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.395960 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.395967 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.395974 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.395981 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.395989 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.395996 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt' 2026-01-21 11:38:24.396004 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.396012 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.396020 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.396043 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.396052 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.396059 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.396066 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.396074 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.396081 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.396088 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.396102 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.396110 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.396117 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.396124 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.396145 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.396154 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.396162 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.396170 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.396178 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.396185 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.396193 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.396201 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.396209 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.396216 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.396223 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.396231 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.396241 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.396261 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.396272 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.396280 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.396289 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.396297 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key' 2026-01-21 11:38:24.396304 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.396311 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.396318 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.396325 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.396333 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.396340 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.396347 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.396354 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.396415 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.396426 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.396434 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.396442 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.396450 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.396457 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.396465 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.396473 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.396481 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.396489 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.396497 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.396505 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.396512 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.396519 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.396526 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.396534 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.396552 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.396561 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.396569 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.396576 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.396584 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.396592 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.396600 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.396608 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt' 2026-01-21 11:38:24.396616 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.396624 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.396648 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.396656 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.396664 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.396672 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.396680 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.396706 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.396734 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.396747 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.396754 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.396762 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.396769 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.396777 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.396785 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.396809 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.396819 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.396828 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.396836 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.396844 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.396852 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.396860 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.396867 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.396875 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.396883 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.396890 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.396898 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.396907 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.396915 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.396923 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.396930 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.396938 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-applier-0 2026-01-21 11:38:24.396946 | controller | '']''' 2026-01-21 11:38:24.396969 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-applier 2026-01-21 11:38:24.396978 | controller | config...''' 2026-01-21 11:38:24.396986 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | Checking watcher-applier 2026-01-21 11:38:24.396994 | controller | config...' 2026-01-21 11:38:24.397002 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.397010 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.397018 | controller | - ' logger.go:42: 11:24:37 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.397030 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.397038 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.397044 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.397051 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.397060 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.397068 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.397076 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.397084 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397092 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.397099 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.397120 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.397130 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.397138 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.397146 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.397153 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.397160 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.397186 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.397196 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397204 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.397222 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.397231 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.397238 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.397245 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.397253 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.397260 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.397268 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.397276 | controller | period = 900' 2026-01-21 11:38:24.397283 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.397291 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_keyfile 2026-01-21 11:38:24.397299 | controller | = /etc/pki/tls/private/mtls.key''' 2026-01-21 11:38:24.397323 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.397329 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.397342 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.397350 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.397359 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.397375 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.397383 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.397391 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.397398 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397406 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.397413 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.397421 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.397428 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.397433 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.397440 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.397447 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.397455 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.397463 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.397470 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397478 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.397483 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.397489 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.397495 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.397503 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.397510 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.397518 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.397540 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.397550 | controller | period = 900' 2026-01-21 11:38:24.397558 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.397566 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_cafile 2026-01-21 11:38:24.397574 | controller | = /etc/pki/tls/certs/mtls-ca.crt''' 2026-01-21 11:38:24.397597 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.397604 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.397612 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.397619 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.397627 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.397635 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.397642 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.397648 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.397654 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397660 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.397667 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.397674 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.397682 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.397690 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.397730 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.397745 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.397753 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.397761 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.397767 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397774 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.397798 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.397808 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.397815 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.397823 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.397831 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.397839 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.397846 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.397853 | controller | period = 900' 2026-01-21 11:38:24.397859 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.397865 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_enabled 2026-01-21 11:38:24.397871 | controller | = true''' 2026-01-21 11:38:24.397883 | controller | - ' logger.go:42: 11:24:38 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.397890 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.397898 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.397906 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.397926 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.397947 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.397956 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.397964 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.397972 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.397980 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.397987 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.397993 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.398001 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398008 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398016 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.398024 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398032 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398039 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.398046 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398052 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.398059 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.398067 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.398075 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.398082 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.398090 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.398098 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.398105 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.398111 | controller | period = 900' 2026-01-21 11:38:24.398117 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.398125 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt' 2026-01-21 11:38:24.398133 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.398173 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.398183 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.398190 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.398197 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.398216 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.398225 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.398234 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.398243 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398251 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.398260 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.398269 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.398289 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398297 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398303 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.398309 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398315 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398321 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.398327 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398334 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.398341 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.398349 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.398357 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.398365 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.398373 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.398379 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.398384 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.398392 | controller | period = 900' 2026-01-21 11:38:24.398399 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.398406 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key' 2026-01-21 11:38:24.398412 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.398426 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.398433 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.398442 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.398459 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.398468 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.398475 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.398482 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.398489 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398498 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.398505 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.398513 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.398519 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398530 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398536 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.398542 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398550 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398557 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.398565 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398591 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.398601 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.398621 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.398630 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.398636 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.398642 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.398650 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.398658 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.398683 | controller | period = 900' 2026-01-21 11:38:24.398691 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.398700 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt' 2026-01-21 11:38:24.398708 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.398749 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.398758 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.398770 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.398778 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.398786 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.398794 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.398801 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.398808 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398815 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.398823 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.398831 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.398838 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398845 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398852 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.398860 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.398868 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.398876 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.398882 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.398888 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.398893 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.398899 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.398905 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.398913 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.398920 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.398928 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.398936 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.398943 | controller | period = 900' 2026-01-21 11:38:24.398949 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.398955 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.398963 | controller | '']''' 2026-01-21 11:38:24.398985 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-decision-engine 2026-01-21 11:38:24.398998 | controller | config...''' 2026-01-21 11:38:24.399006 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | Checking watcher-decision-engine 2026-01-21 11:38:24.399014 | controller | config...' 2026-01-21 11:38:24.399022 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.399029 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.399035 | controller | - ' logger.go:42: 11:24:39 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.399042 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.399049 | controller | - ' logger.go:42: 11:24:41 | watcher-tls/2-patch-mtls | error: Internal error occurred: 2026-01-21 11:38:24.399057 | controller | error executing command in container: container is not created or running' 2026-01-21 11:38:24.399065 | controller | - ' logger.go:42: 11:24:41 | watcher-tls/2-patch-mtls | ++ echo' 2026-01-21 11:38:24.399072 | controller | - ' logger.go:42: 11:24:41 | watcher-tls/2-patch-mtls | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.399078 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | running command: [sh -c 2026-01-21 11:38:24.399085 | controller | set -euxo pipefail' 2026-01-21 11:38:24.399092 | controller | - ' ' 2026-01-21 11:38:24.399100 | controller | - ' oc project ${NAMESPACE}' 2026-01-21 11:38:24.399108 | controller | - ' # Get pod names for each watcher service' 2026-01-21 11:38:24.399116 | controller | - ' APIPOD=$(oc get pods -l service=watcher-api -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.399122 | controller | - ' APPLIERPOD=$(oc get pods -l service=watcher-applier -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.399128 | controller | - ' DECISIONENGINEPOD=$(oc get pods -l service=watcher-decision-engine -o jsonpath=''{.items[0].metadata.name}'')' 2026-01-21 11:38:24.399135 | controller | - ' ' 2026-01-21 11:38:24.399143 | controller | - ' # Verify memcached mTLS config parameters in watcher-api config' 2026-01-21 11:38:24.399151 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.399159 | controller | - ' echo "Checking watcher-api config..."' 2026-01-21 11:38:24.399165 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399172 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.399180 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399190 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.399197 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399204 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.399210 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399217 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.399224 | controller | - ' ' 2026-01-21 11:38:24.399232 | controller | - ' # Verify mTLS config parameters in memcached backend config' 2026-01-21 11:38:24.399240 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399247 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.399255 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399262 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.399270 | controller | - ' [ $(echo $(oc rsh -c watcher-api $APIPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399278 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.399285 | controller | - ' else' 2026-01-21 11:38:24.399306 | controller | - ' exit 1' 2026-01-21 11:38:24.399315 | controller | - ' fi' 2026-01-21 11:38:24.399323 | controller | - ' ' 2026-01-21 11:38:24.399331 | controller | - ' # Verify memcached mTLS config parameters in watcher-applier config' 2026-01-21 11:38:24.399338 | controller | - ' if [ -n "${APPLIERPOD}" ]; then' 2026-01-21 11:38:24.399346 | controller | - ' echo "Checking watcher-applier config..."' 2026-01-21 11:38:24.399352 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399358 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.399366 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399374 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.399382 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399389 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.399396 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399403 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.399411 | controller | - ' ' 2026-01-21 11:38:24.399419 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399439 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.399447 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399455 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.399464 | controller | - ' [ $(echo $(oc rsh -c watcher-applier $APPLIERPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399472 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.399477 | controller | - ' else' 2026-01-21 11:38:24.399484 | controller | - ' exit 1' 2026-01-21 11:38:24.399491 | controller | - ' fi' 2026-01-21 11:38:24.399499 | controller | - ' ' 2026-01-21 11:38:24.399507 | controller | - ' # Verify memcached mTLS config parameters in watcher-decision-engine config' 2026-01-21 11:38:24.399515 | controller | - ' if [ -n "${DECISIONENGINEPOD}" ]; then' 2026-01-21 11:38:24.399522 | controller | - ' echo "Checking watcher-decision-engine config..."' 2026-01-21 11:38:24.399530 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399538 | controller | | grep -c "memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt ") == 1 ]' 2026-01-21 11:38:24.399546 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399552 | controller | | grep -c "memcache_tls_keyfile = /etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.399558 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399565 | controller | | grep -c "memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.399572 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399580 | controller | | grep -c "memcache_tls_enabled = true") == 1 ]' 2026-01-21 11:38:24.399588 | controller | - ' ' 2026-01-21 11:38:24.399595 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399601 | controller | | grep -c "tls_certfile=/etc/pki/tls/certs/mtls.crt") == 1 ]' 2026-01-21 11:38:24.399622 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399642 | controller | | grep -c "tls_keyfile=/etc/pki/tls/private/mtls.key") == 1 ]' 2026-01-21 11:38:24.399649 | controller | - ' [ $(echo $(oc rsh -c watcher-decision-engine $DECISIONENGINEPOD cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.399655 | controller | | grep -c "tls_cafile=/etc/pki/tls/certs/mtls-ca.crt") == 1 ]' 2026-01-21 11:38:24.399662 | controller | - ' else' 2026-01-21 11:38:24.399670 | controller | - ' exit 1' 2026-01-21 11:38:24.399678 | controller | - ' fi' 2026-01-21 11:38:24.399686 | controller | - ' ]' 2026-01-21 11:38:24.399692 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.399698 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | Already on project "watcher-kuttl-default" 2026-01-21 11:38:24.399705 | controller | on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.399735 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-api 2026-01-21 11:38:24.399747 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.399755 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.399762 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-applier 2026-01-21 11:38:24.399770 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.399778 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | + APPLIERPOD=watcher-kuttl-applier-0' 2026-01-21 11:38:24.399784 | controller | - ' logger.go:42: 11:24:42 | watcher-tls/2-patch-mtls | ++ oc get pods -l service=watcher-decision-engine 2026-01-21 11:38:24.399790 | controller | -o ''jsonpath={.items[0].metadata.name}''' 2026-01-21 11:38:24.399797 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | + DECISIONENGINEPOD=watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.399804 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-api-0 2026-01-21 11:38:24.399821 | controller | '']''' 2026-01-21 11:38:24.399828 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-api 2026-01-21 11:38:24.399838 | controller | config...''' 2026-01-21 11:38:24.399846 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | Checking watcher-api config...' 2026-01-21 11:38:24.399854 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.399861 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.399868 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.399874 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.399881 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.399902 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.399912 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.399920 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.399937 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.399943 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.399949 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.399955 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.399965 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.399971 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.399979 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.399996 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400006 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400014 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.400020 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400026 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400032 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.400038 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400044 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.400049 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.400055 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.400061 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.400067 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.400073 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.400079 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.400084 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.400090 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.400096 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.400102 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.400108 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_keyfile 2026-01-21 11:38:24.400116 | controller | = /etc/pki/tls/private/mtls.key''' 2026-01-21 11:38:24.400124 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.400132 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.400140 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.400148 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.400155 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.400161 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.400167 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.400174 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.400187 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.400207 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400215 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.400223 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.400233 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.400251 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400259 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400267 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.400273 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400280 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400288 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.400294 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400300 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.400307 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.400316 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.400324 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.400331 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.400339 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.400347 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.400362 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.400368 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.400374 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.400380 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.400386 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_cafile 2026-01-21 11:38:24.400392 | controller | = /etc/pki/tls/certs/mtls-ca.crt''' 2026-01-21 11:38:24.400398 | controller | - ' logger.go:42: 11:24:43 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.400404 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.400411 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.400424 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.400432 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.400440 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.400447 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.400458 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.400464 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.400470 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400476 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.400482 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.400488 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.400493 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400499 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400505 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.400523 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400530 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400538 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.400546 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400554 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.400578 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.400585 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.400592 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.400600 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.400608 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.400616 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.400623 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.400631 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.400639 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.400647 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.400655 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_enabled 2026-01-21 11:38:24.400662 | controller | = true''' 2026-01-21 11:38:24.400670 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.400678 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.400685 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.400705 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.400743 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.400756 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.400762 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.400768 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.400774 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.400780 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400786 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.400792 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.400798 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.400804 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400812 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400819 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.400827 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.400833 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.400839 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.400846 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.400854 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.400862 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.400883 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.400890 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.400896 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.400902 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.400908 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.400914 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.400920 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.400926 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.400932 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.400939 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt' 2026-01-21 11:38:24.400947 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.400954 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.400960 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.400982 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.400991 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.400999 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.401024 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.401044 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.401053 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.401060 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401099 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.401110 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.401117 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.401123 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401131 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401138 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.401145 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401152 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401158 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.401164 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401171 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.401177 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.401183 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.401191 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.401198 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.401205 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.401212 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.401218 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.401224 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.401242 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.401250 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.401256 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key' 2026-01-21 11:38:24.401272 | controller | - ' logger.go:42: 11:24:44 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.401284 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.401290 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.401296 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.401302 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.401308 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.401314 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.401319 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.401325 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.401331 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401337 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.401343 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.401349 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.401362 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401368 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401374 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.401380 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401386 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401392 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.401398 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401416 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.401422 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.401428 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.401434 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.401439 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.401445 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.401451 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.401457 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.401463 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.401469 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.401475 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.401488 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt' 2026-01-21 11:38:24.401494 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-api 2026-01-21 11:38:24.401500 | controller | watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.401506 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.401522 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.401528 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.401534 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.401540 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.401546 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.401552 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.401559 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401565 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.401571 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.401576 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.401582 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401588 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401594 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.401600 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401606 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401613 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.401620 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401628 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.401636 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.401654 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.401663 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.401671 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.401679 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.401686 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.401692 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.401698 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.401731 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.401744 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.401764 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-applier-0 2026-01-21 11:38:24.401773 | controller | '']''' 2026-01-21 11:38:24.401780 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-applier 2026-01-21 11:38:24.401785 | controller | config...''' 2026-01-21 11:38:24.401792 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | Checking watcher-applier 2026-01-21 11:38:24.401797 | controller | config...' 2026-01-21 11:38:24.401805 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.401812 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.401821 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.401828 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.401835 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.401841 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.401847 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.401854 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.401876 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.401884 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.401891 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401899 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.401906 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.401913 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.401919 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401925 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401931 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.401937 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.401942 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.401948 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.401954 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.401965 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.401973 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.401985 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.401999 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.402030 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.402044 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.402051 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.402058 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.402065 | controller | period = 900' 2026-01-21 11:38:24.402073 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.402081 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_keyfile 2026-01-21 11:38:24.402088 | controller | = /etc/pki/tls/private/mtls.key''' 2026-01-21 11:38:24.402094 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.402100 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.402106 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.402112 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.402118 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.402124 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.402130 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.402136 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.402142 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.402147 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.402153 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.402159 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.402165 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.402171 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.402194 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.402201 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.402207 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.402213 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.402218 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.402224 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.402231 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.402239 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.402245 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.402255 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.402262 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.402270 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.402276 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.402283 | controller | period = 900' 2026-01-21 11:38:24.402290 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.402296 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_cafile 2026-01-21 11:38:24.402302 | controller | = /etc/pki/tls/certs/mtls-ca.crt''' 2026-01-21 11:38:24.402308 | controller | - ' logger.go:42: 11:24:45 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.402314 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.402320 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.402326 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.402332 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.402337 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.402343 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.402349 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.402356 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.402362 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.402368 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.402373 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.402379 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.402386 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.402394 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.402412 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.402438 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.402449 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.402457 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.402465 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.402472 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.402480 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.402509 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.402518 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.402526 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.402534 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.402541 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.402548 | controller | period = 900' 2026-01-21 11:38:24.402556 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.402564 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_enabled 2026-01-21 11:38:24.402572 | controller | = true''' 2026-01-21 11:38:24.402614 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.402623 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.402631 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.402639 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.402646 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.402653 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.402661 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.402669 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.402676 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.402683 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.402691 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.402698 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.402706 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.402767 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.402782 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.402807 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.402815 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.402823 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.402830 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.402838 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.402846 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.402854 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.402869 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.402883 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.402891 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.402899 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.402907 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.402915 | controller | period = 900' 2026-01-21 11:38:24.402923 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.402931 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt' 2026-01-21 11:38:24.402938 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.402946 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.402972 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.402987 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.403002 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.403009 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.403016 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.403040 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.403048 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403056 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.403064 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.403071 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.403078 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403086 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403093 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.403101 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403108 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403116 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.403131 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403142 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.403149 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.403155 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.403161 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.403172 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.403178 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.403184 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.403190 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.403196 | controller | period = 900' 2026-01-21 11:38:24.403202 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.403208 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key' 2026-01-21 11:38:24.403213 | controller | - ' logger.go:42: 11:24:46 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.403219 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.403225 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.403231 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.403237 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.403243 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.403249 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.403255 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.403261 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403281 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.403288 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.403294 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.403300 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403306 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403312 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.403318 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403331 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403337 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.403343 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403349 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.403355 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.403361 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.403367 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.403376 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.403382 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.403388 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.403394 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.403400 | controller | period = 900' 2026-01-21 11:38:24.403406 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.403412 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt' 2026-01-21 11:38:24.403418 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-applier 2026-01-21 11:38:24.403423 | controller | watcher-kuttl-applier-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.403429 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.403435 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.403441 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.403447 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.403453 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.403459 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.403465 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403470 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.403479 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.403485 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.403491 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403497 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403503 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.403509 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403515 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403521 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.403538 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403545 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.403551 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.403557 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[cinder_client]'' 2026-01-21 11:38:24.403562 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.403576 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.403582 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.403588 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.403594 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.403600 | controller | period = 900' 2026-01-21 11:38:24.403605 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.403611 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + ''['' -n watcher-kuttl-decision-engine-0 2026-01-21 11:38:24.403617 | controller | '']''' 2026-01-21 11:38:24.403623 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + echo ''Checking watcher-decision-engine 2026-01-21 11:38:24.403629 | controller | config...''' 2026-01-21 11:38:24.403635 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | Checking watcher-decision-engine 2026-01-21 11:38:24.403641 | controller | config...' 2026-01-21 11:38:24.403647 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.403653 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.403659 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_certfile 2026-01-21 11:38:24.403664 | controller | = /etc/pki/tls/certs/mtls.crt ''' 2026-01-21 11:38:24.403671 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.403677 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.403682 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.403688 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.403694 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.403700 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.403706 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403732 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.403739 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.403745 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.403751 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403757 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403763 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.403769 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403775 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403781 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.403787 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403797 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.403804 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.403809 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.403828 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.403835 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.403840 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.403846 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.403852 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.403858 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.403864 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.403870 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.403876 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_keyfile 2026-01-21 11:38:24.403882 | controller | = /etc/pki/tls/private/mtls.key''' 2026-01-21 11:38:24.403892 | controller | - ' logger.go:42: 11:24:47 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.403898 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.403904 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.403910 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.403915 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.403921 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.403927 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.403933 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.403939 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.403945 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.403951 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.403957 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.403963 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.403970 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.403981 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.403997 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404006 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404028 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.404036 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404042 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.404048 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.404054 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.404060 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.404066 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.404072 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.404083 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.404089 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.404095 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.404101 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.404107 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.404128 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_cafile 2026-01-21 11:38:24.404136 | controller | = /etc/pki/tls/certs/mtls-ca.crt''' 2026-01-21 11:38:24.404142 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.404148 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.404154 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.404160 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.404167 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.404173 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.404179 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.404186 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.404194 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404200 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.404205 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.404211 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.404217 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404223 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404229 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.404235 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404245 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404254 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.404260 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404266 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.404272 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.404278 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.404284 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.404290 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.404296 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.404302 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.404308 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.404314 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.404320 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.404326 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.404331 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ grep -c ''memcache_tls_enabled 2026-01-21 11:38:24.404337 | controller | = true''' 2026-01-21 11:38:24.404343 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.404349 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.404355 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.404361 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.404367 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.404385 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.404391 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.404397 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.404403 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404409 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.404415 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.404421 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.404427 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404433 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404449 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.404455 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404461 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404467 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.404473 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404479 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.404485 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.404490 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.404496 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.404502 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.404508 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.404514 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.404520 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.404526 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.404534 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.404540 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.404546 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ grep -c tls_certfile=/etc/pki/tls/certs/mtls.crt' 2026-01-21 11:38:24.404552 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.404558 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.404564 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.404570 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.404578 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.404587 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.404593 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.404599 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.404604 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404610 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.404616 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.404630 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.404637 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404648 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404654 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.404661 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404666 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404672 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.404678 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404684 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.404690 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.404696 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.404702 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.404708 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.404733 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.404739 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.404745 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.404751 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.404757 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.404763 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.404768 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | ++ grep -c tls_keyfile=/etc/pki/tls/private/mtls.key' 2026-01-21 11:38:24.404774 | controller | - ' logger.go:42: 11:24:48 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.404780 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.404786 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.404792 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.404798 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.404804 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.404810 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.404816 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.404821 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404827 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.404833 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.404839 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.404848 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404854 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404860 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.404866 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.404872 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.404878 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.404893 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.404899 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.404905 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.404911 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.404917 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.404923 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.404929 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.404935 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.404941 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.404946 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.404953 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.404958 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.404965 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | +++ oc rsh -c watcher-decision-engine 2026-01-21 11:38:24.404972 | controller | watcher-kuttl-decision-engine-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.404980 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | ++ grep -c tls_cafile=/etc/pki/tls/certs/mtls-ca.crt' 2026-01-21 11:38:24.404988 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | ++ echo ''[DEFAULT]'' state_path 2026-01-21 11:38:24.404996 | controller | = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.405003 | controller | control_exchange = watcher debug = True ''#'' empty notification_level means that 2026-01-21 11:38:24.405010 | controller | no notification will be sent notification_level = ''[database]'' connection = ''mysql+pymysql://watcher_9466:2ce6bf28ff16d6ee0764b2f6f62a3dc6@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.405018 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.405025 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.405032 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.405045 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.405056 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.405063 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.405071 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.405078 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.405085 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.405092 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.405099 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.405107 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.405114 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.405125 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.405132 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.405139 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.405147 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.405154 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.405161 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.405183 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.405191 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.405198 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.405206 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.405213 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.405220 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/2-patch-mtls | test step completed 2-patch-mtls' 2026-01-21 11:38:24.405228 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | starting test 2026-01-21 11:38:24.405237 | controller | step 3-disable-podlevel-tls' 2026-01-21 11:38:24.405245 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.405253 | controller | [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type=''json'' -p=''[{"op": 2026-01-21 11:38:24.405262 | controller | "replace", "path": "/spec/apiServiceTemplate/tls/api", "value":{ "internal": {}, 2026-01-21 11:38:24.405271 | controller | "public": {} }}]''' 2026-01-21 11:38:24.405279 | controller | - ' ]' 2026-01-21 11:38:24.405288 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.405296 | controller | patched' 2026-01-21 11:38:24.405309 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.405317 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.405324 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.405331 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.405339 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.405347 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.405360 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.405368 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.405376 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.405383 | controller | ]' 2026-01-21 11:38:24.405405 | controller | - ' # check that watcher internal endpoint does not use https' 2026-01-21 11:38:24.405413 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.405420 | controller | infra-optim | grep internal | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.405428 | controller | - ' # check that watcher public endpoint does use https' 2026-01-21 11:38:24.405436 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.405444 | controller | infra-optim | grep public | [ $(grep -c https) == 1 ]' 2026-01-21 11:38:24.405452 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.405460 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.405468 | controller | == "" ]; then' 2026-01-21 11:38:24.405475 | controller | - ' exit 0' 2026-01-21 11:38:24.405483 | controller | - ' fi' 2026-01-21 11:38:24.405490 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.405498 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.405505 | controller | - ' counter=0' 2026-01-21 11:38:24.405512 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.405520 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.405528 | controller | - ' echo ${i}' 2026-01-21 11:38:24.405536 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.405543 | controller | - ' fi' 2026-01-21 11:38:24.405551 | controller | - ' done' 2026-01-21 11:38:24.405558 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.405566 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.405573 | controller | - ' exit 1' 2026-01-21 11:38:24.405581 | controller | - ' else' 2026-01-21 11:38:24.405588 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.405595 | controller | - ' fi' 2026-01-21 11:38:24.405603 | controller | - ' ]' 2026-01-21 11:38:24.405610 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.405617 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.405625 | controller | -c Type' 2026-01-21 11:38:24.405632 | controller | - ' logger.go:42: 11:24:49 | watcher-tls/3-disable-podlevel-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.405640 | controller | - ' logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.405647 | controller | 1 '']''' 2026-01-21 11:38:24.405655 | controller | - ' logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls | ++ grep watcher' 2026-01-21 11:38:24.405662 | controller | - ' logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls | ++ awk ''{print 2026-01-21 11:38:24.405670 | controller | $1}''' 2026-01-21 11:38:24.405677 | controller | - ' logger.go:42: 11:24:53 | watcher-tls/3-disable-podlevel-tls | ++ oc exec -n 2026-01-21 11:38:24.405685 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.405692 | controller | -c Type -c ID' 2026-01-21 11:38:24.405730 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.405745 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.405752 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.405764 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.405777 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.405784 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.405791 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.405798 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.405806 | controller | '']''' 2026-01-21 11:38:24.405813 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + grep internal' 2026-01-21 11:38:24.405820 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.405828 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.405836 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.405846 | controller | - ' logger.go:42: 11:24:56 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.405853 | controller | - ' logger.go:42: 11:24:59 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.405860 | controller | 0 '']''' 2026-01-21 11:38:24.405868 | controller | - ' logger.go:42: 11:25:00 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.405875 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.405882 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.405889 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.405897 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.405904 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.405911 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.405919 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.405927 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.405934 | controller | ]' 2026-01-21 11:38:24.405942 | controller | - ' # check that watcher internal endpoint does not use https' 2026-01-21 11:38:24.405949 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.405957 | controller | infra-optim | grep internal | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.405965 | controller | - ' # check that watcher public endpoint does use https' 2026-01-21 11:38:24.405973 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.405980 | controller | infra-optim | grep public | [ $(grep -c https) == 1 ]' 2026-01-21 11:38:24.405988 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.405995 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.406003 | controller | == "" ]; then' 2026-01-21 11:38:24.406017 | controller | - ' exit 0' 2026-01-21 11:38:24.406025 | controller | - ' fi' 2026-01-21 11:38:24.406033 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.406040 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.406048 | controller | - ' counter=0' 2026-01-21 11:38:24.406055 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.406063 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.406071 | controller | - ' echo ${i}' 2026-01-21 11:38:24.406078 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.406091 | controller | - ' fi' 2026-01-21 11:38:24.406098 | controller | - ' done' 2026-01-21 11:38:24.406106 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.406113 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.406121 | controller | - ' exit 1' 2026-01-21 11:38:24.406129 | controller | - ' else' 2026-01-21 11:38:24.406136 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.406144 | controller | - ' fi' 2026-01-21 11:38:24.406151 | controller | - ' ]' 2026-01-21 11:38:24.406158 | controller | - ' logger.go:42: 11:25:00 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.406166 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.406174 | controller | -c Type' 2026-01-21 11:38:24.406183 | controller | - ' logger.go:42: 11:25:00 | watcher-tls/3-disable-podlevel-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.406192 | controller | - ' logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.406199 | controller | 1 '']''' 2026-01-21 11:38:24.406207 | controller | - ' logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls | ++ oc exec -n 2026-01-21 11:38:24.406214 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.406222 | controller | -c Type -c ID' 2026-01-21 11:38:24.406230 | controller | - ' logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls | ++ grep watcher' 2026-01-21 11:38:24.406252 | controller | - ' logger.go:42: 11:25:02 | watcher-tls/3-disable-podlevel-tls | ++ awk ''{print 2026-01-21 11:38:24.406262 | controller | $1}''' 2026-01-21 11:38:24.406270 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.406278 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.406286 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.406294 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.406301 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.406309 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.406317 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.406325 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.406333 | controller | '']''' 2026-01-21 11:38:24.406340 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.406348 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + grep internal' 2026-01-21 11:38:24.406355 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.406363 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.406370 | controller | - ' logger.go:42: 11:25:05 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.406378 | controller | - ' logger.go:42: 11:25:08 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.406386 | controller | 0 '']''' 2026-01-21 11:38:24.406394 | controller | - ' logger.go:42: 11:25:09 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.406401 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.406408 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.406415 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.406422 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.406429 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.406453 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.406462 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.406470 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.406477 | controller | ]' 2026-01-21 11:38:24.406484 | controller | - ' # check that watcher internal endpoint does not use https' 2026-01-21 11:38:24.406491 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.406499 | controller | infra-optim | grep internal | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.406507 | controller | - ' # check that watcher public endpoint does use https' 2026-01-21 11:38:24.406514 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.406521 | controller | infra-optim | grep public | [ $(grep -c https) == 1 ]' 2026-01-21 11:38:24.406528 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.406536 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.406543 | controller | == "" ]; then' 2026-01-21 11:38:24.406550 | controller | - ' exit 0' 2026-01-21 11:38:24.406558 | controller | - ' fi' 2026-01-21 11:38:24.406565 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.406608 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.406619 | controller | - ' counter=0' 2026-01-21 11:38:24.406626 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.406634 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.406642 | controller | - ' echo ${i}' 2026-01-21 11:38:24.406655 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.406662 | controller | - ' fi' 2026-01-21 11:38:24.406670 | controller | - ' done' 2026-01-21 11:38:24.406677 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.406685 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.406692 | controller | - ' exit 1' 2026-01-21 11:38:24.406700 | controller | - ' else' 2026-01-21 11:38:24.406708 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.406740 | controller | - ' fi' 2026-01-21 11:38:24.406749 | controller | - ' ]' 2026-01-21 11:38:24.406757 | controller | - ' logger.go:42: 11:25:09 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.406771 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.406779 | controller | -c Type' 2026-01-21 11:38:24.406787 | controller | - ' logger.go:42: 11:25:09 | watcher-tls/3-disable-podlevel-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.406795 | controller | - ' logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.406803 | controller | 1 '']''' 2026-01-21 11:38:24.406810 | controller | - ' logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls | ++ oc exec -n 2026-01-21 11:38:24.406818 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.406826 | controller | -c Type -c ID' 2026-01-21 11:38:24.406852 | controller | - ' logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls | ++ awk ''{print 2026-01-21 11:38:24.406862 | controller | $1}''' 2026-01-21 11:38:24.406870 | controller | - ' logger.go:42: 11:25:11 | watcher-tls/3-disable-podlevel-tls | ++ grep watcher' 2026-01-21 11:38:24.406878 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.406886 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.406894 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.406912 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.406921 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.406929 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.406936 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.406945 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.406953 | controller | '']''' 2026-01-21 11:38:24.406961 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + grep internal' 2026-01-21 11:38:24.406969 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.406976 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.406984 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.406992 | controller | - ' logger.go:42: 11:25:14 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.407000 | controller | - ' logger.go:42: 11:25:17 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.407008 | controller | 0 '']''' 2026-01-21 11:38:24.407016 | controller | - ' logger.go:42: 11:25:18 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.407024 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.407032 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.407039 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.407047 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.407055 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.407062 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.407070 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.407078 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.407086 | controller | ]' 2026-01-21 11:38:24.407093 | controller | - ' # check that watcher internal endpoint does not use https' 2026-01-21 11:38:24.407101 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.407109 | controller | infra-optim | grep internal | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.407117 | controller | - ' # check that watcher public endpoint does use https' 2026-01-21 11:38:24.407126 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.407134 | controller | infra-optim | grep public | [ $(grep -c https) == 1 ]' 2026-01-21 11:38:24.407142 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.407149 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.407157 | controller | == "" ]; then' 2026-01-21 11:38:24.407164 | controller | - ' exit 0' 2026-01-21 11:38:24.407172 | controller | - ' fi' 2026-01-21 11:38:24.407179 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.407186 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.407193 | controller | - ' counter=0' 2026-01-21 11:38:24.407201 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.407208 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.407216 | controller | - ' echo ${i}' 2026-01-21 11:38:24.407223 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.407230 | controller | - ' fi' 2026-01-21 11:38:24.407248 | controller | - ' done' 2026-01-21 11:38:24.407256 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.407266 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.407275 | controller | - ' exit 1' 2026-01-21 11:38:24.407283 | controller | - ' else' 2026-01-21 11:38:24.407292 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.407300 | controller | - ' fi' 2026-01-21 11:38:24.407307 | controller | - ' ]' 2026-01-21 11:38:24.407315 | controller | - ' logger.go:42: 11:25:18 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.407322 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.407330 | controller | -c Type' 2026-01-21 11:38:24.407337 | controller | - ' logger.go:42: 11:25:18 | watcher-tls/3-disable-podlevel-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.407345 | controller | - ' logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.407353 | controller | 1 '']''' 2026-01-21 11:38:24.407375 | controller | - ' logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls | ++ oc exec -n 2026-01-21 11:38:24.407384 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.407391 | controller | -c Type -c ID' 2026-01-21 11:38:24.407399 | controller | - ' logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls | ++ awk ''{print 2026-01-21 11:38:24.407406 | controller | $1}''' 2026-01-21 11:38:24.407414 | controller | - ' logger.go:42: 11:25:22 | watcher-tls/3-disable-podlevel-tls | ++ grep watcher' 2026-01-21 11:38:24.407421 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.407428 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.407436 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.407444 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.407451 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.407457 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.407468 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.407475 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.407481 | controller | '']''' 2026-01-21 11:38:24.407486 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.407492 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + grep internal' 2026-01-21 11:38:24.407498 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.407504 | controller | - ' logger.go:42: 11:25:25 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.407510 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.407516 | controller | - ' logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | + ''['' 0 == 2026-01-21 11:38:24.407522 | controller | 0 '']''' 2026-01-21 11:38:24.407527 | controller | - ' logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.407533 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.407539 | controller | - ' logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | + grep public' 2026-01-21 11:38:24.407545 | controller | - ' logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.407551 | controller | - ' logger.go:42: 11:25:29 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.407557 | controller | - ' logger.go:42: 11:25:31 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.407566 | controller | 1 '']''' 2026-01-21 11:38:24.407572 | controller | - ' logger.go:42: 11:25:31 | watcher-tls/3-disable-podlevel-tls | ++ oc get pods 2026-01-21 11:38:24.407578 | controller | -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.407584 | controller | - ' logger.go:42: 11:25:32 | watcher-tls/3-disable-podlevel-tls | + ''['' '''' 2026-01-21 11:38:24.407590 | controller | == '''' '']''' 2026-01-21 11:38:24.407596 | controller | - ' logger.go:42: 11:25:32 | watcher-tls/3-disable-podlevel-tls | + exit 0' 2026-01-21 11:38:24.407601 | controller | - ' logger.go:42: 11:25:33 | watcher-tls/3-disable-podlevel-tls | running command: 2026-01-21 11:38:24.407607 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.407613 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.407619 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.407625 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.407631 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.407636 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.407642 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.407648 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.407654 | controller | ]' 2026-01-21 11:38:24.407660 | controller | - ' # check that watcher internal endpoint does not use https' 2026-01-21 11:38:24.407666 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.407672 | controller | infra-optim | grep internal | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.407677 | controller | - ' # check that watcher public endpoint does use https' 2026-01-21 11:38:24.407683 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.407689 | controller | infra-optim | grep public | [ $(grep -c https) == 1 ]' 2026-01-21 11:38:24.407695 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.407701 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.407707 | controller | == "" ]; then' 2026-01-21 11:38:24.407730 | controller | - ' exit 0' 2026-01-21 11:38:24.407743 | controller | - ' fi' 2026-01-21 11:38:24.407749 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.407766 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.407773 | controller | - ' counter=0' 2026-01-21 11:38:24.407794 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.407800 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.407806 | controller | - ' echo ${i}' 2026-01-21 11:38:24.407812 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.407818 | controller | - ' fi' 2026-01-21 11:38:24.407824 | controller | - ' done' 2026-01-21 11:38:24.407830 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.407836 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.407842 | controller | - ' exit 1' 2026-01-21 11:38:24.407848 | controller | - ' else' 2026-01-21 11:38:24.407854 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.407860 | controller | - ' fi' 2026-01-21 11:38:24.407866 | controller | - ' ]' 2026-01-21 11:38:24.407872 | controller | - ' logger.go:42: 11:25:33 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.407878 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.407884 | controller | -c Type' 2026-01-21 11:38:24.407890 | controller | - ' logger.go:42: 11:25:33 | watcher-tls/3-disable-podlevel-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.407900 | controller | - ' logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.407906 | controller | 1 '']''' 2026-01-21 11:38:24.407912 | controller | - ' logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls | ++ oc exec -n 2026-01-21 11:38:24.407918 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.407924 | controller | -c Type -c ID' 2026-01-21 11:38:24.407930 | controller | - ' logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls | ++ awk ''{print 2026-01-21 11:38:24.407936 | controller | $1}''' 2026-01-21 11:38:24.407942 | controller | - ' logger.go:42: 11:25:36 | watcher-tls/3-disable-podlevel-tls | ++ grep watcher' 2026-01-21 11:38:24.407948 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.407953 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.407959 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.407966 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.407974 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.407981 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | ++ oc get -n 2026-01-21 11:38:24.407989 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.407997 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.408005 | controller | '']''' 2026-01-21 11:38:24.408060 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.408071 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.408077 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + grep internal' 2026-01-21 11:38:24.408083 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.408089 | controller | - ' logger.go:42: 11:25:39 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.408094 | controller | - ' logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls | + ''['' 0 == 2026-01-21 11:38:24.408100 | controller | 0 '']''' 2026-01-21 11:38:24.408106 | controller | - ' logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls | + oc exec -n 2026-01-21 11:38:24.408112 | controller | watcher-kuttl-default openstackclient -- openstack endpoint list' 2026-01-21 11:38:24.408118 | controller | - ' logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls | + grep infra-optim' 2026-01-21 11:38:24.408124 | controller | - ' logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls | + grep public' 2026-01-21 11:38:24.408130 | controller | - ' logger.go:42: 11:25:42 | watcher-tls/3-disable-podlevel-tls | ++ grep -c https' 2026-01-21 11:38:24.408136 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | + ''['' 1 == 2026-01-21 11:38:24.408142 | controller | 1 '']''' 2026-01-21 11:38:24.408147 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | ++ oc get pods 2026-01-21 11:38:24.408153 | controller | -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.408159 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | + ''['' '''' 2026-01-21 11:38:24.408165 | controller | == '''' '']''' 2026-01-21 11:38:24.408171 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | + exit 0' 2026-01-21 11:38:24.408178 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/3-disable-podlevel-tls | test step completed 2026-01-21 11:38:24.408185 | controller | 3-disable-podlevel-tls' 2026-01-21 11:38:24.408191 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | starting test 2026-01-21 11:38:24.408208 | controller | step 4-deploy-without-route' 2026-01-21 11:38:24.408214 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | running command: 2026-01-21 11:38:24.408220 | controller | [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type=''json'' -p=''[{"op": 2026-01-21 11:38:24.408236 | controller | "replace", "path": "/spec/apiServiceTemplate/override", "value":{"service": { "internal": 2026-01-21 11:38:24.408242 | controller | {}, "public": { "metadata": { "annotations": { "metallb.universe.tf/address-pool": 2026-01-21 11:38:24.408248 | controller | "ctlplane", "metallb.universe.tf/allow-shared-ip": "ctlplane" } }, "spec": { "type": 2026-01-21 11:38:24.408254 | controller | "LoadBalancer" } } } }}]''' 2026-01-21 11:38:24.408260 | controller | - ' ]' 2026-01-21 11:38:24.408266 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.408272 | controller | patched' 2026-01-21 11:38:24.408278 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | running command: 2026-01-21 11:38:24.408284 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.408290 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.408296 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.408302 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.408308 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.408345 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.408351 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.408357 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.408363 | controller | ]' 2026-01-21 11:38:24.408369 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.408375 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.408381 | controller | == "" ]; then' 2026-01-21 11:38:24.408387 | controller | - ' exit 0' 2026-01-21 11:38:24.408392 | controller | - ' fi' 2026-01-21 11:38:24.408398 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.408404 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.408410 | controller | - ' counter=0' 2026-01-21 11:38:24.408416 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.408422 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.408428 | controller | - ' echo ${i}' 2026-01-21 11:38:24.408434 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.408439 | controller | - ' fi' 2026-01-21 11:38:24.408445 | controller | - ' done' 2026-01-21 11:38:24.408451 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.408457 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.408463 | controller | - ' exit 1' 2026-01-21 11:38:24.408469 | controller | - ' else' 2026-01-21 11:38:24.408475 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.408481 | controller | - ' fi' 2026-01-21 11:38:24.408487 | controller | - ' ]' 2026-01-21 11:38:24.408493 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | + oc exec -n 2026-01-21 11:38:24.408499 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.408505 | controller | -c Type' 2026-01-21 11:38:24.408511 | controller | - ' logger.go:42: 11:25:45 | watcher-tls/4-deploy-without-route | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.408517 | controller | - ' logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route | + ''['' 1 == 2026-01-21 11:38:24.408526 | controller | 1 '']''' 2026-01-21 11:38:24.408532 | controller | - ' logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route | ++ oc exec -n 2026-01-21 11:38:24.408538 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.408544 | controller | -c Type -c ID' 2026-01-21 11:38:24.408552 | controller | - ' logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route | ++ awk ''{print 2026-01-21 11:38:24.408568 | controller | $1}''' 2026-01-21 11:38:24.408581 | controller | - ' logger.go:42: 11:25:48 | watcher-tls/4-deploy-without-route | ++ grep watcher' 2026-01-21 11:38:24.408590 | controller | - ' logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.408597 | controller | - ' logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | ++ oc get -n 2026-01-21 11:38:24.408610 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.408618 | controller | - ' logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.408625 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.408632 | controller | - ' logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | ++ oc get -n 2026-01-21 11:38:24.408639 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.408645 | controller | - ' logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.408651 | controller | '']''' 2026-01-21 11:38:24.408657 | controller | - ' logger.go:42: 11:25:50 | watcher-tls/4-deploy-without-route | ++ oc get pods 2026-01-21 11:38:24.408663 | controller | -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.408669 | controller | - ' logger.go:42: 11:25:51 | watcher-tls/4-deploy-without-route | + ''['' '''' 2026-01-21 11:38:24.408675 | controller | == '''' '']''' 2026-01-21 11:38:24.408681 | controller | - ' logger.go:42: 11:25:51 | watcher-tls/4-deploy-without-route | + exit 0' 2026-01-21 11:38:24.408700 | controller | - ' logger.go:42: 11:25:52 | watcher-tls/4-deploy-without-route | running command: 2026-01-21 11:38:24.408707 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.408733 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.408744 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.408750 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.408759 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.408765 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.408771 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.408777 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.408783 | controller | ]' 2026-01-21 11:38:24.408789 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.408795 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.408801 | controller | == "" ]; then' 2026-01-21 11:38:24.408807 | controller | - ' exit 0' 2026-01-21 11:38:24.408814 | controller | - ' fi' 2026-01-21 11:38:24.408821 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.408829 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.408836 | controller | - ' counter=0' 2026-01-21 11:38:24.408843 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.408851 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.408858 | controller | - ' echo ${i}' 2026-01-21 11:38:24.408871 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.408879 | controller | - ' fi' 2026-01-21 11:38:24.408886 | controller | - ' done' 2026-01-21 11:38:24.408893 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.408900 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.408908 | controller | - ' exit 1' 2026-01-21 11:38:24.408915 | controller | - ' else' 2026-01-21 11:38:24.408922 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.408930 | controller | - ' fi' 2026-01-21 11:38:24.408937 | controller | - ' ]' 2026-01-21 11:38:24.408945 | controller | - ' logger.go:42: 11:25:52 | watcher-tls/4-deploy-without-route | + oc exec -n 2026-01-21 11:38:24.408952 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.408959 | controller | -c Type' 2026-01-21 11:38:24.408967 | controller | - ' logger.go:42: 11:25:52 | watcher-tls/4-deploy-without-route | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.408975 | controller | - ' logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route | + ''['' 1 == 2026-01-21 11:38:24.408982 | controller | 1 '']''' 2026-01-21 11:38:24.408990 | controller | - ' logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route | ++ oc exec -n 2026-01-21 11:38:24.408998 | controller | watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.409005 | controller | -c Type -c ID' 2026-01-21 11:38:24.409012 | controller | - ' logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route | ++ awk ''{print 2026-01-21 11:38:24.409020 | controller | $1}''' 2026-01-21 11:38:24.409027 | controller | - ' logger.go:42: 11:25:54 | watcher-tls/4-deploy-without-route | ++ grep watcher' 2026-01-21 11:38:24.409034 | controller | - ' logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.409042 | controller | - ' logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route | ++ oc get -n 2026-01-21 11:38:24.409049 | controller | watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.409057 | controller | - ' logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.409064 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.409071 | controller | - ' logger.go:42: 11:25:56 | watcher-tls/4-deploy-without-route | ++ oc get -n 2026-01-21 11:38:24.409084 | controller | watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.409092 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.409100 | controller | '']''' 2026-01-21 11:38:24.409107 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | ++ oc get pods 2026-01-21 11:38:24.409115 | controller | -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.409122 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | + ''['' '''' 2026-01-21 11:38:24.409130 | controller | == '''' '']''' 2026-01-21 11:38:24.409138 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | + exit 0' 2026-01-21 11:38:24.409145 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/4-deploy-without-route | test step completed 2026-01-21 11:38:24.409153 | controller | 4-deploy-without-route' 2026-01-21 11:38:24.409160 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | starting test step 5-disable-tls' 2026-01-21 11:38:24.409168 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | running command: [sh -c 2026-01-21 11:38:24.409176 | controller | oc patch watcher -n $NAMESPACE watcher-kuttl --type=''json'' -p=''[{"op": "replace", 2026-01-21 11:38:24.409183 | controller | "path": "/spec/apiServiceTemplate/override", "value":{}}]''' 2026-01-21 11:38:24.409191 | controller | - ' ]' 2026-01-21 11:38:24.409236 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.409248 | controller | patched' 2026-01-21 11:38:24.409255 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | running command: [sh -c 2026-01-21 11:38:24.409263 | controller | set -euxo pipefail' 2026-01-21 11:38:24.409272 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.409280 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.409290 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.409298 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.409305 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.409313 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.409321 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.409329 | controller | ]' 2026-01-21 11:38:24.409336 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.409343 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.409351 | controller | == "" ]; then' 2026-01-21 11:38:24.409359 | controller | - ' exit 0' 2026-01-21 11:38:24.409367 | controller | - ' fi' 2026-01-21 11:38:24.409375 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.409383 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.409391 | controller | - ' counter=0' 2026-01-21 11:38:24.409398 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.409406 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.409413 | controller | - ' echo ${i}' 2026-01-21 11:38:24.409421 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.409429 | controller | - ' fi' 2026-01-21 11:38:24.409437 | controller | - ' done' 2026-01-21 11:38:24.409444 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.409452 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.409460 | controller | - ' exit 1' 2026-01-21 11:38:24.409467 | controller | - ' else' 2026-01-21 11:38:24.409475 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.409482 | controller | - ' fi' 2026-01-21 11:38:24.409490 | controller | - ' # check that no watcher endpoint uses https' 2026-01-21 11:38:24.409499 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.409516 | controller | infra-optim | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.409524 | controller | - ' ]' 2026-01-21 11:38:24.409532 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.409540 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.409549 | controller | - ' logger.go:42: 11:25:57 | watcher-tls/5-disable-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.409557 | controller | - ' logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.409565 | controller | - ' logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.409572 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.409580 | controller | - ' logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | ++ awk ''{print $1}''' 2026-01-21 11:38:24.409587 | controller | - ' logger.go:42: 11:25:59 | watcher-tls/5-disable-tls | ++ grep watcher' 2026-01-21 11:38:24.409594 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.409601 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.409615 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.409623 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.409630 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.409638 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.409645 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.409652 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.409661 | controller | '']''' 2026-01-21 11:38:24.409668 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | ++ oc get pods -n openstack-operators 2026-01-21 11:38:24.409674 | controller | -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.409680 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | + ''['' '''' == '''' '']''' 2026-01-21 11:38:24.409686 | controller | - ' logger.go:42: 11:26:02 | watcher-tls/5-disable-tls | + exit 0' 2026-01-21 11:38:24.409692 | controller | - ' logger.go:42: 11:26:03 | watcher-tls/5-disable-tls | running command: [sh -c 2026-01-21 11:38:24.409698 | controller | set -euxo pipefail' 2026-01-21 11:38:24.409704 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack service list -f value 2026-01-21 11:38:24.409710 | controller | -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.409746 | controller | - ' SERVICEID=$(oc exec -n ${NAMESPACE} openstackclient -- openstack service 2026-01-21 11:38:24.409771 | controller | list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.409780 | controller | - ' [ $(oc get -n ${NAMESPACE} keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.409787 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.409794 | controller | - ' [ -n "$(oc get -n ${NAMESPACE} watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.409802 | controller | ]' 2026-01-21 11:38:24.409810 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.409818 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.409825 | controller | == "" ]; then' 2026-01-21 11:38:24.409833 | controller | - ' exit 0' 2026-01-21 11:38:24.409840 | controller | - ' fi' 2026-01-21 11:38:24.409847 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.409855 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.409863 | controller | - ' counter=0' 2026-01-21 11:38:24.409870 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.409877 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.409883 | controller | - ' echo ${i}' 2026-01-21 11:38:24.409890 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.409897 | controller | - ' fi' 2026-01-21 11:38:24.409904 | controller | - ' done' 2026-01-21 11:38:24.409911 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.409936 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.409946 | controller | - ' exit 1' 2026-01-21 11:38:24.409954 | controller | - ' else' 2026-01-21 11:38:24.409961 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.409968 | controller | - ' fi' 2026-01-21 11:38:24.409975 | controller | - ' # check that no watcher endpoint uses https' 2026-01-21 11:38:24.409982 | controller | - ' oc exec -n ${NAMESPACE} openstackclient -- openstack endpoint list | grep 2026-01-21 11:38:24.409989 | controller | infra-optim | [ $(grep -c https) == 0 ]' 2026-01-21 11:38:24.410002 | controller | - ' ]' 2026-01-21 11:38:24.410010 | controller | - ' logger.go:42: 11:26:03 | watcher-tls/5-disable-tls | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.410018 | controller | - ' logger.go:42: 11:26:03 | watcher-tls/5-disable-tls | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.410025 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.410033 | controller | - ' logger.go:42: 11:26:06 | watcher-tls/5-disable-tls | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.410041 | controller | - ' logger.go:42: 11:26:06 | watcher-tls/5-disable-tls | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.410048 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.410056 | controller | - ' logger.go:42: 11:26:06 | watcher-tls/5-disable-tls | ++ awk ''{print $1}''' 2026-01-21 11:38:24.410063 | controller | - ' logger.go:42: 11:26:06 | watcher-tls/5-disable-tls | ++ grep watcher' 2026-01-21 11:38:24.410071 | controller | - ' logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | + SERVICEID=eee10af9590f44de87d443cfc0f41a86' 2026-01-21 11:38:24.410079 | controller | - ' logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.410087 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.410094 | controller | - ' logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | + ''['' eee10af9590f44de87d443cfc0f41a86 2026-01-21 11:38:24.410102 | controller | == eee10af9590f44de87d443cfc0f41a86 '']''' 2026-01-21 11:38:24.410308 | controller | - ' logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.410326 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.410340 | controller | - ' logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.410349 | controller | '']''' 2026-01-21 11:38:24.410357 | controller | - ' logger.go:42: 11:26:08 | watcher-tls/5-disable-tls | ++ oc get pods -n openstack-operators 2026-01-21 11:38:24.410365 | controller | -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.410373 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/5-disable-tls | + ''['' '''' == '''' '']''' 2026-01-21 11:38:24.410387 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/5-disable-tls | + exit 0' 2026-01-21 11:38:24.410395 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/5-disable-tls | test step completed 5-disable-tls' 2026-01-21 11:38:24.410403 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher | starting test step 2026-01-21 11:38:24.410410 | controller | 6-cleanup-watcher' 2026-01-21 11:38:24.410418 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher | running command: [sh 2026-01-21 11:38:24.410445 | controller | -c set -ex' 2026-01-21 11:38:24.410454 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.410462 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.410470 | controller | - ' ]' 2026-01-21 11:38:24.410478 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.410485 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.410493 | controller | - ' logger.go:42: 11:26:09 | watcher-tls/6-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.410501 | controller | - ' logger.go:42: 11:26:12 | watcher-tls/6-cleanup-watcher | + ''['' 1 == 0 '']''' 2026-01-21 11:38:24.410533 | controller | - ' logger.go:42: 11:26:13 | watcher-tls/6-cleanup-watcher | running command: [sh 2026-01-21 11:38:24.410545 | controller | -c set -ex' 2026-01-21 11:38:24.410553 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.410568 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.410607 | controller | - ' ]' 2026-01-21 11:38:24.410617 | controller | - ' logger.go:42: 11:26:13 | watcher-tls/6-cleanup-watcher | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.410630 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.410639 | controller | - ' logger.go:42: 11:26:13 | watcher-tls/6-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.410647 | controller | - ' logger.go:42: 11:26:17 | watcher-tls/6-cleanup-watcher | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.410654 | controller | - ' logger.go:42: 11:26:17 | watcher-tls/6-cleanup-watcher | test step completed 2026-01-21 11:38:24.410661 | controller | 6-cleanup-watcher' 2026-01-21 11:38:24.410669 | controller | - ' logger.go:42: 11:26:17 | watcher-tls/7-cleanup-certs | starting test step 7-cleanup-certs' 2026-01-21 11:38:24.410676 | controller | - ' logger.go:42: 11:26:17 | watcher-tls/7-cleanup-certs | test step completed 2026-01-21 11:38:24.410684 | controller | 7-cleanup-certs' 2026-01-21 11:38:24.410691 | controller | - ' logger.go:42: 11:26:17 | watcher-tls | skipping kubernetes event logging' 2026-01-21 11:38:24.410699 | controller | - === CONT kuttl/harness/watcher-rmquser 2026-01-21 11:38:24.410706 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser | Skipping creation of user-supplied 2026-01-21 11:38:24.410732 | controller | namespace: watcher-kuttl-default' 2026-01-21 11:38:24.410745 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser/0-cleanup-watcher | starting test 2026-01-21 11:38:24.410759 | controller | step 0-cleanup-watcher' 2026-01-21 11:38:24.410766 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser/0-cleanup-watcher | test step completed 2026-01-21 11:38:24.410774 | controller | 0-cleanup-watcher' 2026-01-21 11:38:24.410781 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser/1-deploy | starting test step 1-deploy' 2026-01-21 11:38:24.410788 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser/1-deploy | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.410796 | controller | created' 2026-01-21 11:38:24.410803 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser/1-deploy | running command: [sh -c 2026-01-21 11:38:24.410810 | controller | set -euxo pipefail' 2026-01-21 11:38:24.410818 | controller | - ' ' 2026-01-21 11:38:24.410826 | controller | - ' # Wait for Watcher to be Ready' 2026-01-21 11:38:24.410833 | controller | - ' kubectl wait --for=condition=Ready watcher/watcher-kuttl -n $NAMESPACE 2026-01-21 11:38:24.410841 | controller | --timeout=300s' 2026-01-21 11:38:24.410849 | controller | - ' ' 2026-01-21 11:38:24.410855 | controller | - ' # Verify WatcherNotificationTransportURLReady condition exists and is True' 2026-01-21 11:38:24.410863 | controller | - ' kubectl get watcher watcher-kuttl -n $NAMESPACE -o jsonpath=''{.status.conditions[?(@.type=="WatcherNotificationTransportURLReady")].status}'' 2026-01-21 11:38:24.410871 | controller | | grep -q "True"' 2026-01-21 11:38:24.410879 | controller | - ' echo "WatcherNotificationTransportURLReady condition is True"' 2026-01-21 11:38:24.410886 | controller | - ' ' 2026-01-21 11:38:24.410894 | controller | - ' # Count TransportURL CRs - should be exactly 2 (one for messaging, one 2026-01-21 11:38:24.410905 | controller | for notifications)' 2026-01-21 11:38:24.410921 | controller | - ' transport_count=$(kubectl get transporturl -n $NAMESPACE -o name | grep 2026-01-21 11:38:24.410929 | controller | "watcher-kuttl-watcher-transport" | wc -l)' 2026-01-21 11:38:24.410937 | controller | - ' notification_transport_count=$(kubectl get transporturl -n $NAMESPACE -o 2026-01-21 11:38:24.410944 | controller | name | grep "watcher-kuttl-watcher-notification" | wc -l)' 2026-01-21 11:38:24.410952 | controller | - ' ' 2026-01-21 11:38:24.410960 | controller | - ' if [ "$transport_count" -ne "1" ]; then' 2026-01-21 11:38:24.410968 | controller | - ' echo "Expected 1 watcher-transport TransportURL, found $transport_count"' 2026-01-21 11:38:24.410976 | controller | - ' exit 1' 2026-01-21 11:38:24.410984 | controller | - ' fi' 2026-01-21 11:38:24.410992 | controller | - ' ' 2026-01-21 11:38:24.411000 | controller | - ' if [ "$notification_transport_count" -ne "1" ]; then' 2026-01-21 11:38:24.411008 | controller | - ' echo "Expected 1 notification-transport TransportURL, found $notification_transport_count"' 2026-01-21 11:38:24.411034 | controller | - ' exit 1' 2026-01-21 11:38:24.411043 | controller | - ' fi' 2026-01-21 11:38:24.411051 | controller | - ' ' 2026-01-21 11:38:24.411059 | controller | - ' echo "Correctly found 2 TransportURLs (separate clusters: transport and 2026-01-21 11:38:24.411067 | controller | notification)"' 2026-01-21 11:38:24.411075 | controller | - ' ' 2026-01-21 11:38:24.411083 | controller | - ' # Verify watcher-transport has correct user and vhost' 2026-01-21 11:38:24.411090 | controller | - ' transport_user=$(kubectl get transporturl watcher-kuttl-watcher-transport 2026-01-21 11:38:24.411098 | controller | -n $NAMESPACE -o jsonpath=''{.spec.username}'')' 2026-01-21 11:38:24.411106 | controller | - ' transport_vhost=$(kubectl get transporturl watcher-kuttl-watcher-transport 2026-01-21 11:38:24.411114 | controller | -n $NAMESPACE -o jsonpath=''{.spec.vhost}'')' 2026-01-21 11:38:24.411121 | controller | - ' if [ "$transport_user" != "watcher-rpc" ]; then' 2026-01-21 11:38:24.411129 | controller | - ' echo "Expected watcher-transport username ''watcher-rpc'', found ''$transport_user''"' 2026-01-21 11:38:24.411137 | controller | - ' exit 1' 2026-01-21 11:38:24.411145 | controller | - ' fi' 2026-01-21 11:38:24.411153 | controller | - ' if [ "$transport_vhost" != "watcher-rpc" ]; then' 2026-01-21 11:38:24.411160 | controller | - ' echo "Expected watcher-transport vhost ''watcher-rpc'', found ''$transport_vhost''"' 2026-01-21 11:38:24.411168 | controller | - ' exit 1' 2026-01-21 11:38:24.411195 | controller | - ' fi' 2026-01-21 11:38:24.411205 | controller | - ' echo "Watcher transport has correct user (watcher-rpc) and vhost (watcher-rpc)"' 2026-01-21 11:38:24.411213 | controller | - ' ' 2026-01-21 11:38:24.411221 | controller | - ' # Verify notification-transport has correct user and vhost' 2026-01-21 11:38:24.411229 | controller | - ' notif_user=$(kubectl get transporturl watcher-kuttl-watcher-notification-rabbitmq-notifications 2026-01-21 11:38:24.411237 | controller | -n $NAMESPACE -o jsonpath=''{.spec.username}'')' 2026-01-21 11:38:24.411261 | controller | - ' notif_vhost=$(kubectl get transporturl watcher-kuttl-watcher-notification-rabbitmq-notifications 2026-01-21 11:38:24.411271 | controller | -n $NAMESPACE -o jsonpath=''{.spec.vhost}'')' 2026-01-21 11:38:24.411279 | controller | - ' if [ "$notif_user" != "watcher-notifications" ]; then' 2026-01-21 11:38:24.411287 | controller | - ' echo "Expected notification-transport username ''watcher-notifications'', 2026-01-21 11:38:24.411295 | controller | found ''$notif_user''"' 2026-01-21 11:38:24.411302 | controller | - ' exit 1' 2026-01-21 11:38:24.411310 | controller | - ' fi' 2026-01-21 11:38:24.411318 | controller | - ' if [ "$notif_vhost" != "watcher-notifications" ]; then' 2026-01-21 11:38:24.411325 | controller | - ' echo "Expected notification-transport vhost ''watcher-notifications'', 2026-01-21 11:38:24.411333 | controller | found ''$notif_vhost''"' 2026-01-21 11:38:24.411341 | controller | - ' exit 1' 2026-01-21 11:38:24.411349 | controller | - ' fi' 2026-01-21 11:38:24.411357 | controller | - ' echo "Notification transport has correct user (watcher-notifications) and 2026-01-21 11:38:24.411365 | controller | vhost (watcher-notifications)"' 2026-01-21 11:38:24.411373 | controller | - ' ' 2026-01-21 11:38:24.411381 | controller | - ' # Verify that watcher.conf contains the notifications transport_url' 2026-01-21 11:38:24.411389 | controller | - ' WATCHER_API_POD=$(kubectl get pods -n $NAMESPACE -l "service=watcher-api" 2026-01-21 11:38:24.411397 | controller | -o custom-columns=:metadata.name --no-headers | grep -v ^$ | head -1)' 2026-01-21 11:38:24.411404 | controller | - ' if [ -z "${WATCHER_API_POD}" ]; then' 2026-01-21 11:38:24.411412 | controller | - ' echo "No watcher-api pod found"' 2026-01-21 11:38:24.411420 | controller | - ' exit 1' 2026-01-21 11:38:24.411428 | controller | - ' fi' 2026-01-21 11:38:24.411436 | controller | - ' # Verify RPC transport_url in DEFAULT section' 2026-01-21 11:38:24.411443 | controller | - ' rpc_transport_url=$(kubectl exec -n $NAMESPACE ${WATCHER_API_POD} -c watcher-api 2026-01-21 11:38:24.411456 | controller | -- cat /etc/watcher/watcher.conf.d/00-default.conf | grep -E ''^\[DEFAULT\]'' -A 2026-01-21 11:38:24.411464 | controller | 50 | grep ''transport_url'' | head -1 || true)' 2026-01-21 11:38:24.411477 | controller | - ' if [ -z "$rpc_transport_url" ]; then' 2026-01-21 11:38:24.411485 | controller | - ' echo "transport_url not found in DEFAULT section"' 2026-01-21 11:38:24.411493 | controller | - ' exit 1' 2026-01-21 11:38:24.411501 | controller | - ' fi' 2026-01-21 11:38:24.411509 | controller | - ' echo "Found RPC transport_url: $rpc_transport_url"' 2026-01-21 11:38:24.411516 | controller | - ' ' 2026-01-21 11:38:24.411524 | controller | - ' # Verify the RPC transport_url contains the correct vhost (watcher-rpc)' 2026-01-21 11:38:24.411532 | controller | - ' if ! echo "$rpc_transport_url" | grep -q ''/watcher-rpc''; then' 2026-01-21 11:38:24.411540 | controller | - ' echo "RPC transport_url does not contain expected vhost ''/watcher-rpc''"' 2026-01-21 11:38:24.411548 | controller | - ' exit 1' 2026-01-21 11:38:24.411555 | controller | - ' fi' 2026-01-21 11:38:24.411563 | controller | - ' echo "Successfully verified vhost ''watcher-rpc'' in RPC transport_url"' 2026-01-21 11:38:24.411571 | controller | - ' ' 2026-01-21 11:38:24.411578 | controller | - ' # Verify the RPC transport_url contains the correct username (watcher-rpc)' 2026-01-21 11:38:24.411586 | controller | - ' if ! echo "$rpc_transport_url" | grep -q ''watcher-rpc:''; then' 2026-01-21 11:38:24.411594 | controller | - ' echo "RPC transport_url does not contain expected username ''watcher-rpc:''"' 2026-01-21 11:38:24.411602 | controller | - ' exit 1' 2026-01-21 11:38:24.411610 | controller | - ' fi' 2026-01-21 11:38:24.411617 | controller | - ' echo "Successfully verified username ''watcher-rpc'' in RPC transport_url"' 2026-01-21 11:38:24.411625 | controller | - ' ' 2026-01-21 11:38:24.411633 | controller | - ' # Verify oslo_messaging_notifications section has transport_url configured' 2026-01-21 11:38:24.411641 | controller | - ' notif_transport_url=$(kubectl exec -n $NAMESPACE ${WATCHER_API_POD} -c 2026-01-21 11:38:24.411648 | controller | watcher-api -- cat /etc/watcher/watcher.conf.d/00-default.conf | grep -A 5 ''\[oslo_messaging_notifications\]'' 2026-01-21 11:38:24.411656 | controller | | grep ''transport_url'' || true)' 2026-01-21 11:38:24.411664 | controller | - ' if [ -z "$notif_transport_url" ]; then' 2026-01-21 11:38:24.411672 | controller | - ' echo "transport_url not found in oslo_messaging_notifications section"' 2026-01-21 11:38:24.411679 | controller | - ' exit 1' 2026-01-21 11:38:24.411688 | controller | - ' fi' 2026-01-21 11:38:24.411695 | controller | - ' echo "Found notifications transport_url: $notif_transport_url"' 2026-01-21 11:38:24.411703 | controller | - ' ' 2026-01-21 11:38:24.411711 | controller | - ' # Verify the notifications transport_url contains the correct vhost (watcher-notifications)' 2026-01-21 11:38:24.411747 | controller | - ' if ! echo "$notif_transport_url" | grep -q ''/watcher-notifications''; 2026-01-21 11:38:24.411755 | controller | then' 2026-01-21 11:38:24.411763 | controller | - ' echo "Notifications transport_url does not contain expected vhost ''/watcher-notifications''"' 2026-01-21 11:38:24.411770 | controller | - ' exit 1' 2026-01-21 11:38:24.411778 | controller | - ' fi' 2026-01-21 11:38:24.411801 | controller | - ' echo "Successfully verified vhost ''watcher-notifications'' in notifications 2026-01-21 11:38:24.411810 | controller | transport_url"' 2026-01-21 11:38:24.411817 | controller | - ' ' 2026-01-21 11:38:24.411840 | controller | - ' # Verify the notifications transport_url contains the correct username 2026-01-21 11:38:24.411849 | controller | (watcher-notifications)' 2026-01-21 11:38:24.411857 | controller | - ' if ! echo "$notif_transport_url" | grep -q ''watcher-notifications:''; 2026-01-21 11:38:24.411865 | controller | then' 2026-01-21 11:38:24.411873 | controller | - ' echo "Notifications transport_url does not contain expected username 2026-01-21 11:38:24.411880 | controller | ''watcher-notifications:''"' 2026-01-21 11:38:24.411889 | controller | - ' exit 1' 2026-01-21 11:38:24.411902 | controller | - ' fi' 2026-01-21 11:38:24.411910 | controller | - ' echo "Successfully verified username ''watcher-notifications'' in notifications 2026-01-21 11:38:24.411918 | controller | transport_url"' 2026-01-21 11:38:24.411925 | controller | - ' ' 2026-01-21 11:38:24.411932 | controller | - ' exit 0' 2026-01-21 11:38:24.411939 | controller | - ' ]' 2026-01-21 11:38:24.411947 | controller | - ' logger.go:42: 11:26:17 | watcher-rmquser/1-deploy | + kubectl wait --for=condition=Ready 2026-01-21 11:38:24.411955 | controller | watcher/watcher-kuttl -n watcher-kuttl-default --timeout=300s' 2026-01-21 11:38:24.411963 | controller | - ' logger.go:42: 11:31:18 | watcher-rmquser/1-deploy | error: timed out waiting 2026-01-21 11:38:24.411970 | controller | for the condition on watchers/watcher-kuttl' 2026-01-21 11:38:24.411978 | controller | - ' logger.go:42: 11:31:18 | watcher-rmquser/1-deploy | test step failed 1-deploy' 2026-01-21 11:38:24.411986 | controller | - ' case.go:396: failed in step 1-deploy' 2026-01-21 11:38:24.411993 | controller | - ' case.go:398: transporturls.rabbitmq.openstack.org "watcher-kuttl-watcher-transport" 2026-01-21 11:38:24.412001 | controller | not found' 2026-01-21 11:38:24.412009 | controller | - ' case.go:398: transporturls.rabbitmq.openstack.org "watcher-kuttl-watcher-notification-rabbitmq-notifications" 2026-01-21 11:38:24.412017 | controller | not found' 2026-01-21 11:38:24.412030 | controller | - ' case.go:398: command "kubectl wait --for=condition=Ready watcher/watcher-kuttl 2026-01-21 11:38:24.412038 | controller | -n $NAMESP..." exceeded 300 sec timeout, context deadline exceeded' 2026-01-21 11:38:24.412046 | controller | - ' logger.go:42: 11:31:18 | watcher-rmquser | skipping kubernetes event logging' 2026-01-21 11:38:24.412054 | controller | - === CONT kuttl/harness/watcher-notification 2026-01-21 11:38:24.412062 | controller | - ' logger.go:42: 11:31:18 | watcher-notification | Skipping creation of user-supplied 2026-01-21 11:38:24.412070 | controller | namespace: watcher-kuttl-default' 2026-01-21 11:38:24.412078 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/0-cleanup-watcher | starting 2026-01-21 11:38:24.412085 | controller | test step 0-cleanup-watcher' 2026-01-21 11:38:24.412093 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/0-cleanup-watcher | test step 2026-01-21 11:38:24.412101 | controller | completed 0-cleanup-watcher' 2026-01-21 11:38:24.412108 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412116 | controller | starting test step 1-deploy-with-notification' 2026-01-21 11:38:24.412124 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412132 | controller | Watcher:watcher-kuttl-default/watcher-kuttl created' 2026-01-21 11:38:24.412140 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412147 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.412155 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.412163 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.412171 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.412179 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.412186 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.412194 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.412202 | controller | == 1 ]' 2026-01-21 11:38:24.412210 | controller | - ' else' 2026-01-21 11:38:24.412217 | controller | - ' exit 1' 2026-01-21 11:38:24.412225 | controller | - ' fi' 2026-01-21 11:38:24.412234 | controller | - ' ]' 2026-01-21 11:38:24.412243 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412251 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.412259 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412291 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.412299 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412308 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.412315 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412323 | controller | ++ head -1' 2026-01-21 11:38:24.412331 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412339 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.412347 | controller | - ' logger.go:42: 11:31:18 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412354 | controller | + APIPOD=' 2026-01-21 11:38:24.412363 | controller | - ' logger.go:42: 11:31:19 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412370 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.412378 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.412386 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.412394 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.412401 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.412424 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.412432 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.412440 | controller | == 1 ]' 2026-01-21 11:38:24.412448 | controller | - ' else' 2026-01-21 11:38:24.412456 | controller | - ' exit 1' 2026-01-21 11:38:24.412463 | controller | - ' fi' 2026-01-21 11:38:24.412471 | controller | - ' ]' 2026-01-21 11:38:24.412479 | controller | - ' logger.go:42: 11:31:19 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412487 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.412495 | controller | - ' logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412502 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.412510 | controller | - ' logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412518 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.412525 | controller | - ' logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412533 | controller | ++ head -1' 2026-01-21 11:38:24.412546 | controller | - ' logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412554 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.412562 | controller | - ' logger.go:42: 11:31:20 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412570 | controller | + APIPOD=' 2026-01-21 11:38:24.412577 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412585 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.412592 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.412600 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.412608 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.412616 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.412624 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.412632 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.412640 | controller | == 1 ]' 2026-01-21 11:38:24.412647 | controller | - ' else' 2026-01-21 11:38:24.412655 | controller | - ' exit 1' 2026-01-21 11:38:24.412662 | controller | - ' fi' 2026-01-21 11:38:24.412688 | controller | - ' ]' 2026-01-21 11:38:24.412697 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412705 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.412744 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412760 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.412768 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412777 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.412785 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412793 | controller | ++ head -1' 2026-01-21 11:38:24.412801 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412808 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.412816 | controller | - ' logger.go:42: 11:31:21 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412824 | controller | + APIPOD=' 2026-01-21 11:38:24.412832 | controller | - ' logger.go:42: 11:31:22 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412839 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.412847 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.412855 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.412863 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.412871 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.412879 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.412886 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.412894 | controller | == 1 ]' 2026-01-21 11:38:24.412902 | controller | - ' else' 2026-01-21 11:38:24.412910 | controller | - ' exit 1' 2026-01-21 11:38:24.412918 | controller | - ' fi' 2026-01-21 11:38:24.412925 | controller | - ' ]' 2026-01-21 11:38:24.412933 | controller | - ' logger.go:42: 11:31:22 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412940 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.412947 | controller | - ' logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412955 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.412963 | controller | - ' logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412970 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.412978 | controller | - ' logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.412985 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.412993 | controller | - ' logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413001 | controller | ++ head -1' 2026-01-21 11:38:24.413008 | controller | - ' logger.go:42: 11:31:23 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413015 | controller | + APIPOD=' 2026-01-21 11:38:24.413042 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413051 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.413059 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.413067 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.413074 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.413082 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.413090 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.413108 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.413117 | controller | == 1 ]' 2026-01-21 11:38:24.413124 | controller | - ' else' 2026-01-21 11:38:24.413132 | controller | - ' exit 1' 2026-01-21 11:38:24.413140 | controller | - ' fi' 2026-01-21 11:38:24.413147 | controller | - ' ]' 2026-01-21 11:38:24.413155 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413163 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.413171 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413179 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.413187 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413194 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.413202 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413210 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.413218 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413226 | controller | ++ head -1' 2026-01-21 11:38:24.413233 | controller | - ' logger.go:42: 11:31:24 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413240 | controller | + APIPOD=' 2026-01-21 11:38:24.413248 | controller | - ' logger.go:42: 11:31:25 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413256 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.413264 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.413272 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.413279 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.413293 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.413301 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.413310 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.413317 | controller | == 1 ]' 2026-01-21 11:38:24.413325 | controller | - ' else' 2026-01-21 11:38:24.413333 | controller | - ' exit 1' 2026-01-21 11:38:24.413341 | controller | - ' fi' 2026-01-21 11:38:24.413348 | controller | - ' ]' 2026-01-21 11:38:24.413356 | controller | - ' logger.go:42: 11:31:25 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413364 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.413372 | controller | - ' logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413380 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.413388 | controller | - ' logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413396 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.413404 | controller | - ' logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413412 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.413420 | controller | - ' logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413428 | controller | ++ head -1' 2026-01-21 11:38:24.413436 | controller | - ' logger.go:42: 11:31:26 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413444 | controller | + APIPOD=' 2026-01-21 11:38:24.413452 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413460 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.413467 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.413475 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.413502 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.413511 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.413520 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.413528 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.413536 | controller | == 1 ]' 2026-01-21 11:38:24.413544 | controller | - ' else' 2026-01-21 11:38:24.413553 | controller | - ' exit 1' 2026-01-21 11:38:24.413560 | controller | - ' fi' 2026-01-21 11:38:24.413568 | controller | - ' ]' 2026-01-21 11:38:24.413577 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413585 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.413599 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413606 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.413614 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413641 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.413650 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413659 | controller | ++ head -1' 2026-01-21 11:38:24.413668 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413676 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.413684 | controller | - ' logger.go:42: 11:31:27 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413693 | controller | + APIPOD=' 2026-01-21 11:38:24.413701 | controller | - ' logger.go:42: 11:31:28 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413710 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.413747 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.413757 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.413765 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.413773 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.413782 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.413790 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.413798 | controller | == 1 ]' 2026-01-21 11:38:24.413805 | controller | - ' else' 2026-01-21 11:38:24.413813 | controller | - ' exit 1' 2026-01-21 11:38:24.413821 | controller | - ' fi' 2026-01-21 11:38:24.413829 | controller | - ' ]' 2026-01-21 11:38:24.413837 | controller | - ' logger.go:42: 11:31:28 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413845 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.413854 | controller | - ' logger.go:42: 11:31:28 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413862 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.413870 | controller | - ' logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413878 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.413886 | controller | - ' logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413894 | controller | ++ head -1' 2026-01-21 11:38:24.413902 | controller | - ' logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413910 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.413918 | controller | - ' logger.go:42: 11:31:29 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413926 | controller | + APIPOD=' 2026-01-21 11:38:24.413939 | controller | - ' logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.413948 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.413956 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.413964 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.413972 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.413980 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.413988 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.413995 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.414003 | controller | == 1 ]' 2026-01-21 11:38:24.414011 | controller | - ' else' 2026-01-21 11:38:24.414020 | controller | - ' exit 1' 2026-01-21 11:38:24.414028 | controller | - ' fi' 2026-01-21 11:38:24.414036 | controller | - ' ]' 2026-01-21 11:38:24.414045 | controller | - ' logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414053 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.414061 | controller | - ' logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414069 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.414077 | controller | - ' logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414085 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.414098 | controller | - ' logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414108 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.414116 | controller | - ' logger.go:42: 11:31:30 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414124 | controller | ++ head -1' 2026-01-21 11:38:24.414133 | controller | - ' logger.go:42: 11:31:31 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414141 | controller | + APIPOD=' 2026-01-21 11:38:24.414153 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414161 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.414169 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.414177 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.414187 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.414195 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.414203 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.414211 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.414219 | controller | == 1 ]' 2026-01-21 11:38:24.414227 | controller | - ' else' 2026-01-21 11:38:24.414236 | controller | - ' exit 1' 2026-01-21 11:38:24.414244 | controller | - ' fi' 2026-01-21 11:38:24.414252 | controller | - ' ]' 2026-01-21 11:38:24.414260 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414286 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.414296 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414303 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.414311 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414318 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.414326 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414338 | controller | ++ head -1' 2026-01-21 11:38:24.414346 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414353 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.414361 | controller | - ' logger.go:42: 11:31:32 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414368 | controller | + APIPOD=' 2026-01-21 11:38:24.414376 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414384 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.414392 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.414400 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.414408 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.414415 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.414422 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.414430 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.414438 | controller | == 1 ]' 2026-01-21 11:38:24.414446 | controller | - ' else' 2026-01-21 11:38:24.414455 | controller | - ' exit 1' 2026-01-21 11:38:24.414462 | controller | - ' fi' 2026-01-21 11:38:24.414470 | controller | - ' ]' 2026-01-21 11:38:24.414478 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414485 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.414493 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414501 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.414508 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414516 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.414524 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414532 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.414540 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414548 | controller | ++ head -1' 2026-01-21 11:38:24.414555 | controller | - ' logger.go:42: 11:31:33 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414563 | controller | + APIPOD=' 2026-01-21 11:38:24.414572 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414599 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.414608 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.414616 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.414630 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.414639 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.414647 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.414655 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.414663 | controller | == 1 ]' 2026-01-21 11:38:24.414671 | controller | - ' else' 2026-01-21 11:38:24.414679 | controller | - ' exit 1' 2026-01-21 11:38:24.414687 | controller | - ' fi' 2026-01-21 11:38:24.414694 | controller | - ' ]' 2026-01-21 11:38:24.414702 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414710 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.414744 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414753 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.414761 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414776 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.414784 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414792 | controller | ++ head -1' 2026-01-21 11:38:24.414800 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414807 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.414815 | controller | - ' logger.go:42: 11:31:35 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414823 | controller | + APIPOD=' 2026-01-21 11:38:24.414831 | controller | - ' logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414839 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.414847 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.414860 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.414868 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.414876 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.414900 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.414911 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.414920 | controller | == 1 ]' 2026-01-21 11:38:24.414929 | controller | - ' else' 2026-01-21 11:38:24.414939 | controller | - ' exit 1' 2026-01-21 11:38:24.414948 | controller | - ' fi' 2026-01-21 11:38:24.414957 | controller | - ' ]' 2026-01-21 11:38:24.414965 | controller | - ' logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414974 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.414983 | controller | - ' logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.414992 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.415000 | controller | - ' logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415008 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.415016 | controller | - ' logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415023 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.415031 | controller | - ' logger.go:42: 11:31:36 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415039 | controller | ++ head -1' 2026-01-21 11:38:24.415047 | controller | - ' logger.go:42: 11:31:37 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415055 | controller | + APIPOD=' 2026-01-21 11:38:24.415062 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415070 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.415079 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.415087 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.415095 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.415102 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.415110 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.415118 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.415126 | controller | == 1 ]' 2026-01-21 11:38:24.415134 | controller | - ' else' 2026-01-21 11:38:24.415143 | controller | - ' exit 1' 2026-01-21 11:38:24.415151 | controller | - ' fi' 2026-01-21 11:38:24.415159 | controller | - ' ]' 2026-01-21 11:38:24.415167 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415187 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.415196 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415205 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.415214 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415227 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.415237 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415245 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.415254 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415264 | controller | ++ head -1' 2026-01-21 11:38:24.415273 | controller | - ' logger.go:42: 11:31:38 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415281 | controller | + APIPOD=' 2026-01-21 11:38:24.415290 | controller | - ' logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415298 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.415305 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.415313 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.415322 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.415331 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.415339 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.415348 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.415357 | controller | == 1 ]' 2026-01-21 11:38:24.415365 | controller | - ' else' 2026-01-21 11:38:24.415374 | controller | - ' exit 1' 2026-01-21 11:38:24.415382 | controller | - ' fi' 2026-01-21 11:38:24.415389 | controller | - ' ]' 2026-01-21 11:38:24.415397 | controller | - ' logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415405 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.415414 | controller | - ' logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415423 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.415431 | controller | - ' logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415440 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.415449 | controller | - ' logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415457 | controller | ++ head -1' 2026-01-21 11:38:24.415465 | controller | - ' logger.go:42: 11:31:39 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415473 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.415481 | controller | - ' logger.go:42: 11:31:40 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415490 | controller | + APIPOD=' 2026-01-21 11:38:24.415515 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415527 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.415536 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.415544 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.415553 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.415560 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.415568 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.415576 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.415591 | controller | == 1 ]' 2026-01-21 11:38:24.415600 | controller | - ' else' 2026-01-21 11:38:24.415608 | controller | - ' exit 1' 2026-01-21 11:38:24.415616 | controller | - ' fi' 2026-01-21 11:38:24.415631 | controller | - ' ]' 2026-01-21 11:38:24.415640 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415648 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.415656 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415665 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.415676 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415685 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.415694 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415702 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.415709 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415743 | controller | ++ head -1' 2026-01-21 11:38:24.415751 | controller | - ' logger.go:42: 11:31:41 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415759 | controller | + APIPOD=' 2026-01-21 11:38:24.415767 | controller | - ' logger.go:42: 11:31:42 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415775 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.415783 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.415791 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.415799 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.415807 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.415816 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.415824 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.415832 | controller | == 1 ]' 2026-01-21 11:38:24.415840 | controller | - ' else' 2026-01-21 11:38:24.415849 | controller | - ' exit 1' 2026-01-21 11:38:24.415858 | controller | - ' fi' 2026-01-21 11:38:24.415867 | controller | - ' ]' 2026-01-21 11:38:24.415875 | controller | - ' logger.go:42: 11:31:42 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415883 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.415891 | controller | - ' logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415899 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.415907 | controller | - ' logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415915 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.415923 | controller | - ' logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415931 | controller | ++ head -1' 2026-01-21 11:38:24.415939 | controller | - ' logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415946 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.415954 | controller | - ' logger.go:42: 11:31:43 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415961 | controller | + APIPOD=' 2026-01-21 11:38:24.415969 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.415976 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.415983 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.415991 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.415998 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.416012 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.416019 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.416027 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.416034 | controller | == 1 ]' 2026-01-21 11:38:24.416041 | controller | - ' else' 2026-01-21 11:38:24.416049 | controller | - ' exit 1' 2026-01-21 11:38:24.416056 | controller | - ' fi' 2026-01-21 11:38:24.416063 | controller | - ' ]' 2026-01-21 11:38:24.416071 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416078 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.416085 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416092 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.416099 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416123 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.416131 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416140 | controller | ++ head -1' 2026-01-21 11:38:24.416147 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416156 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.416164 | controller | - ' logger.go:42: 11:31:44 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416173 | controller | + APIPOD=' 2026-01-21 11:38:24.416191 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416201 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.416211 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.416220 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.416231 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.416241 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.416251 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.416261 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.416270 | controller | == 1 ]' 2026-01-21 11:38:24.416279 | controller | - ' else' 2026-01-21 11:38:24.416288 | controller | - ' exit 1' 2026-01-21 11:38:24.416296 | controller | - ' fi' 2026-01-21 11:38:24.416305 | controller | - ' ]' 2026-01-21 11:38:24.416313 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416321 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.416331 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416340 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.416348 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416357 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.416366 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416374 | controller | ++ head -1' 2026-01-21 11:38:24.416383 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416391 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.416398 | controller | - ' logger.go:42: 11:31:46 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416405 | controller | + APIPOD=' 2026-01-21 11:38:24.416418 | controller | - ' logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416431 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.416438 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.416446 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.416454 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.416461 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.416469 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.416477 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.416485 | controller | == 1 ]' 2026-01-21 11:38:24.416493 | controller | - ' else' 2026-01-21 11:38:24.416502 | controller | - ' exit 1' 2026-01-21 11:38:24.416511 | controller | - ' fi' 2026-01-21 11:38:24.416519 | controller | - ' ]' 2026-01-21 11:38:24.416528 | controller | - ' logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416536 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.416545 | controller | - ' logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416553 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.416562 | controller | - ' logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416570 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.416579 | controller | - ' logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416587 | controller | ++ head -1' 2026-01-21 11:38:24.416596 | controller | - ' logger.go:42: 11:31:47 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416605 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.416612 | controller | - ' logger.go:42: 11:31:48 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416620 | controller | + APIPOD=' 2026-01-21 11:38:24.416627 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416635 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.416642 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.416650 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.416657 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.416665 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.416673 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.416680 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.416688 | controller | == 1 ]' 2026-01-21 11:38:24.416695 | controller | - ' else' 2026-01-21 11:38:24.416703 | controller | - ' exit 1' 2026-01-21 11:38:24.416710 | controller | - ' fi' 2026-01-21 11:38:24.416743 | controller | - ' ]' 2026-01-21 11:38:24.416766 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416776 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.416783 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416791 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.416798 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416806 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.416814 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416821 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.416829 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416842 | controller | ++ head -1' 2026-01-21 11:38:24.416849 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416856 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.416864 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416871 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.416878 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416886 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.416893 | controller | - ' logger.go:42: 11:31:49 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416901 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.416908 | controller | - ' logger.go:42: 11:31:50 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416915 | controller | error: unable to upgrade connection: container not found ("watcher-api")' 2026-01-21 11:38:24.416923 | controller | - ' logger.go:42: 11:31:50 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416931 | controller | ++ echo' 2026-01-21 11:38:24.416938 | controller | - ' logger.go:42: 11:31:50 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416945 | controller | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.416953 | controller | - ' logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.416960 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.416968 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.416976 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.416983 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.416991 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.416999 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.417007 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.417015 | controller | == 1 ]' 2026-01-21 11:38:24.417022 | controller | - ' else' 2026-01-21 11:38:24.417030 | controller | - ' exit 1' 2026-01-21 11:38:24.417038 | controller | - ' fi' 2026-01-21 11:38:24.417046 | controller | - ' ]' 2026-01-21 11:38:24.417054 | controller | - ' logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417062 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.417069 | controller | - ' logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417077 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.417085 | controller | - ' logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417093 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.417106 | controller | - ' logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417113 | controller | ++ head -1' 2026-01-21 11:38:24.417121 | controller | - ' logger.go:42: 11:31:51 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417128 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.417135 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417143 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.417150 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417156 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.417162 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417168 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.417178 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417187 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.417193 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417199 | controller | error: unable to upgrade connection: container not found ("watcher-api")' 2026-01-21 11:38:24.417205 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417211 | controller | ++ echo' 2026-01-21 11:38:24.417217 | controller | - ' logger.go:42: 11:31:52 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417223 | controller | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.417246 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417254 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.417261 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.417268 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.417275 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.417282 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.417289 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.417295 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.417301 | controller | == 1 ]' 2026-01-21 11:38:24.417307 | controller | - ' else' 2026-01-21 11:38:24.417313 | controller | - ' exit 1' 2026-01-21 11:38:24.417319 | controller | - ' fi' 2026-01-21 11:38:24.417325 | controller | - ' ]' 2026-01-21 11:38:24.417331 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417337 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.417343 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417349 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.417355 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417361 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.417367 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417373 | controller | ++ head -1' 2026-01-21 11:38:24.417379 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417385 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.417391 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417397 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.417403 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417409 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.417415 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417420 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.417426 | controller | - ' logger.go:42: 11:31:53 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417432 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.417438 | controller | - ' logger.go:42: 11:31:54 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417445 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.417455 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.417461 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.417468 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.417474 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.417480 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.417487 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.417492 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.417499 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.417505 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.417511 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.417517 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.417523 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.417529 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.417535 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.417541 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.417562 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.417571 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.417579 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.417587 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.417595 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.417602 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.417610 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.417618 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.417631 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.417639 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.417645 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.417651 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.417657 | controller | - ' logger.go:42: 11:31:54 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417663 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.417668 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417674 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.417687 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.417693 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.417699 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.417705 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.417711 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.417736 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.417743 | controller | == 1 ]' 2026-01-21 11:38:24.417748 | controller | - ' else' 2026-01-21 11:38:24.417754 | controller | - ' exit 1' 2026-01-21 11:38:24.417760 | controller | - ' fi' 2026-01-21 11:38:24.417766 | controller | - ' ]' 2026-01-21 11:38:24.417772 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417778 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.417784 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417790 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.417796 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417802 | controller | ++ head -1' 2026-01-21 11:38:24.417808 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417813 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.417819 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417825 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.417831 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417837 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.417843 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417849 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.417855 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417861 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.417867 | controller | - ' logger.go:42: 11:31:55 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417873 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.417878 | controller | - ' logger.go:42: 11:31:56 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.417884 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.417891 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.417897 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.417902 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.417908 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.417925 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.417931 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.417937 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.417947 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.417953 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.417959 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.417965 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.417971 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.417981 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.417987 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.417993 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.417999 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.418005 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.418011 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.418017 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.418022 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.418028 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.418034 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.418043 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.418049 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.418055 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.418061 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.418067 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.418073 | controller | - ' logger.go:42: 11:31:56 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418079 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.418084 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418090 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.418096 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.418102 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.418108 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.418117 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.418123 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.418129 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.418135 | controller | == 1 ]' 2026-01-21 11:38:24.418140 | controller | - ' else' 2026-01-21 11:38:24.418146 | controller | - ' exit 1' 2026-01-21 11:38:24.418153 | controller | - ' fi' 2026-01-21 11:38:24.418158 | controller | - ' ]' 2026-01-21 11:38:24.418164 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418170 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.418181 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418187 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.418193 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418199 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.418205 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418211 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.418217 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418222 | controller | ++ head -1' 2026-01-21 11:38:24.418228 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418235 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.418252 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418262 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.418271 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418279 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.418288 | controller | - ' logger.go:42: 11:31:57 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418296 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.418305 | controller | - ' logger.go:42: 11:31:58 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418313 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.418322 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.418330 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.418338 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.418346 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.418354 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.418362 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.418370 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.418378 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.418385 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.418393 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.418402 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.418409 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.418417 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.418426 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.418441 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.418450 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.418459 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.418467 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.418475 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.418482 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.418490 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.418497 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.418504 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.418511 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.418518 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.418526 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.418532 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.418538 | controller | - ' logger.go:42: 11:31:58 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418545 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.418553 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418560 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.418568 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.418576 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.418620 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.418628 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.418634 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.418640 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.418646 | controller | == 1 ]' 2026-01-21 11:38:24.418652 | controller | - ' else' 2026-01-21 11:38:24.418658 | controller | - ' exit 1' 2026-01-21 11:38:24.418664 | controller | - ' fi' 2026-01-21 11:38:24.418670 | controller | - ' ]' 2026-01-21 11:38:24.418676 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418682 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.418688 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418694 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.418700 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418706 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.418726 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418737 | controller | ++ head -1' 2026-01-21 11:38:24.418743 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418749 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.418758 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418764 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.418775 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418781 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.418790 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418795 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.418802 | controller | - ' logger.go:42: 11:31:59 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418808 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.418814 | controller | - ' logger.go:42: 11:32:00 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.418820 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.418826 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.418831 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.418837 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.418843 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.418849 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.418855 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.418861 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.418867 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.418873 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.418879 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.418885 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.418891 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.418900 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.418906 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.418912 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.418918 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.418924 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.418930 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.418946 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.418953 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.418959 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.418965 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.418975 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.419002 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.419008 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.419014 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.419020 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.419026 | controller | - ' logger.go:42: 11:32:00 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419032 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.419038 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419044 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.419050 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.419056 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.419062 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.419068 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.419074 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.419080 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.419086 | controller | == 1 ]' 2026-01-21 11:38:24.419092 | controller | - ' else' 2026-01-21 11:38:24.419098 | controller | - ' exit 1' 2026-01-21 11:38:24.419104 | controller | - ' fi' 2026-01-21 11:38:24.419110 | controller | - ' ]' 2026-01-21 11:38:24.419116 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419122 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.419128 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419134 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.419140 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419146 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.419152 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419158 | controller | ++ head -1' 2026-01-21 11:38:24.419164 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419170 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.419177 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419184 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.419190 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419196 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.419202 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419208 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.419214 | controller | - ' logger.go:42: 11:32:01 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419220 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.419226 | controller | - ' logger.go:42: 11:32:02 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419231 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.419248 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.419254 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.419260 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.419266 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.419272 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.419278 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.419299 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.419306 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.419312 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.419318 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.419324 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.419331 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.419337 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.419342 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.419348 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.419354 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.419360 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.419370 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.419375 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.419381 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.419387 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.419393 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.419399 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.419405 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.419411 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.419417 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.419423 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.419429 | controller | - ' logger.go:42: 11:32:02 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419437 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.419445 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419453 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.419474 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.419483 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.419491 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.419498 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.419507 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.419514 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.419522 | controller | == 1 ]' 2026-01-21 11:38:24.419530 | controller | - ' else' 2026-01-21 11:38:24.419543 | controller | - ' exit 1' 2026-01-21 11:38:24.419551 | controller | - ' fi' 2026-01-21 11:38:24.419559 | controller | - ' ]' 2026-01-21 11:38:24.419566 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419574 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.419581 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419589 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.419597 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419605 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.419612 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419620 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.419628 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419635 | controller | ++ head -1' 2026-01-21 11:38:24.419642 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419650 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.419657 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419664 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.419671 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419678 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.419696 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419703 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.419709 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419736 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.419743 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.419751 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.419758 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.419766 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.419774 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.419781 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.419789 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.419803 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.419809 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.419815 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.419820 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.419828 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.419835 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.419841 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.419847 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.419853 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.419859 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.419865 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.419870 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.419876 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.419882 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.419892 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.419901 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.419907 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.419913 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.419919 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.419925 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.419930 | controller | - ' logger.go:42: 11:32:03 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419936 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.419942 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.419948 | controller | running command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.419954 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.419960 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.419966 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.419971 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.419979 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.419986 | controller | |grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+='') 2026-01-21 11:38:24.420012 | controller | == 1 ]' 2026-01-21 11:38:24.420022 | controller | - ' else' 2026-01-21 11:38:24.420030 | controller | - ' exit 1' 2026-01-21 11:38:24.420037 | controller | - ' fi' 2026-01-21 11:38:24.420045 | controller | - ' ]' 2026-01-21 11:38:24.420053 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420061 | controller | + oc project watcher-kuttl-default' 2026-01-21 11:38:24.420072 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420080 | controller | Already on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.420087 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420094 | controller | ++ oc get pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.420102 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420108 | controller | ++ grep -v ''^$''' 2026-01-21 11:38:24.420114 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420120 | controller | ++ head -1' 2026-01-21 11:38:24.420126 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420131 | controller | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.420137 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420143 | controller | + ''['' -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.420149 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420155 | controller | ++ grep -czPo ''\[oslo_messaging_notifications\]\s+driver\s+=\s+messagingv2\s+transport_url\s+=''' 2026-01-21 11:38:24.420161 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420167 | controller | +++ oc rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.420172 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420178 | controller | ++ echo ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.420184 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.420190 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.420196 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_17b2:bb4db15cd144e33a790d7ea7ff42ef8f@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.420206 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.420212 | controller | driver = messagingv2 transport_url = ''rabbit://default_user_0VWC4yOh54BP4nixWb3:9dKPXHvc-BijrFR2c6DJV8MMwL-vOAAK@rabbitmq-notifications.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.420217 | controller | ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.420223 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.420230 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.420239 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.420245 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.420251 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.420257 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.420264 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.420271 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.420277 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.420289 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.420295 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.420301 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.420307 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.420312 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.420318 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.420338 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/ca.crt ''[cinder_client]'' endpoint_type 2026-01-21 11:38:24.420345 | controller | = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' endpoint_type 2026-01-21 11:38:24.420351 | controller | = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' endpoint_type 2026-01-21 11:38:24.420357 | controller | = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' interface 2026-01-21 11:38:24.420362 | controller | = internal ''[watcher_cluster_data_model_collectors.compute]'' period = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' 2026-01-21 11:38:24.420369 | controller | period = 900 ''[watcher_cluster_data_model_collectors.storage]'' period = 900' 2026-01-21 11:38:24.420375 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420381 | controller | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.420387 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/1-deploy-with-notification | 2026-01-21 11:38:24.420393 | controller | test step completed 1-deploy-with-notification' 2026-01-21 11:38:24.420402 | controller | - ' logger.go:42: 11:32:05 | watcher-notification/2-cleanup-watcher | starting 2026-01-21 11:38:24.420408 | controller | test step 2-cleanup-watcher' 2026-01-21 11:38:24.420414 | controller | - ' logger.go:42: 11:32:16 | watcher-notification/2-cleanup-watcher | test step 2026-01-21 11:38:24.420419 | controller | completed 2-cleanup-watcher' 2026-01-21 11:38:24.420425 | controller | - ' logger.go:42: 11:32:16 | watcher-notification | skipping kubernetes event logging' 2026-01-21 11:38:24.420431 | controller | - === CONT kuttl/harness/watcher-cinder 2026-01-21 11:38:24.420437 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder | Skipping creation of user-supplied 2026-01-21 11:38:24.420443 | controller | namespace: watcher-kuttl-default' 2026-01-21 11:38:24.420449 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/0-cleanup-watcher | starting test step 2026-01-21 11:38:24.420455 | controller | 0-cleanup-watcher' 2026-01-21 11:38:24.420461 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/0-cleanup-watcher | test step completed 2026-01-21 11:38:24.420467 | controller | 0-cleanup-watcher' 2026-01-21 11:38:24.420473 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | starting 2026-01-21 11:38:24.420479 | controller | test step 1-deploy-watcher-no-cinder' 2026-01-21 11:38:24.420484 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.420490 | controller | created' 2026-01-21 11:38:24.420496 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.420502 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.420508 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.420514 | controller | service' 2026-01-21 11:38:24.420534 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.420540 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.420552 | controller | - ' ]' 2026-01-21 11:38:24.420560 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.420568 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.420575 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.420581 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.420587 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.420593 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.420599 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.420605 | controller | - ' logger.go:42: 11:32:16 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.420611 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.420616 | controller | - ' logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.420622 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.420628 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.420634 | controller | service' 2026-01-21 11:38:24.420640 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.420646 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.420652 | controller | - ' ]' 2026-01-21 11:38:24.420658 | controller | - ' logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.420664 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.420670 | controller | - ' logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.420676 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.420682 | controller | - ' logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.420688 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.420694 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.420700 | controller | - ' logger.go:42: 11:32:18 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.420706 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.420726 | controller | - ' logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.420736 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.420755 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.420762 | controller | service' 2026-01-21 11:38:24.420768 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.420774 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.420780 | controller | - ' ]' 2026-01-21 11:38:24.420786 | controller | - ' logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.420792 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.420798 | controller | - ' logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.420804 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.420810 | controller | - ' logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.420816 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.420822 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.420828 | controller | - ' logger.go:42: 11:32:19 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.420834 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.420844 | controller | - ' logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.420862 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.420868 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.420874 | controller | service' 2026-01-21 11:38:24.420880 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.420886 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.420892 | controller | - ' ]' 2026-01-21 11:38:24.420898 | controller | - ' logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.420904 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.420910 | controller | - ' logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.420916 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.420921 | controller | - ' logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.420927 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.420933 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.420939 | controller | - ' logger.go:42: 11:32:20 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.420945 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.420951 | controller | - ' logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.420957 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.420963 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.420969 | controller | service' 2026-01-21 11:38:24.420975 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.420981 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.420987 | controller | - ' ]' 2026-01-21 11:38:24.420992 | controller | - ' logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.420998 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421004 | controller | - ' logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421010 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421016 | controller | - ' logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421022 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421028 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421034 | controller | - ' logger.go:42: 11:32:21 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421040 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421046 | controller | - ' logger.go:42: 11:32:22 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421051 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421057 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421063 | controller | service' 2026-01-21 11:38:24.421069 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421075 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421081 | controller | - ' ]' 2026-01-21 11:38:24.421087 | controller | - ' logger.go:42: 11:32:23 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421093 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421099 | controller | - ' logger.go:42: 11:32:23 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421105 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421111 | controller | - ' logger.go:42: 11:32:24 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421120 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421126 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421140 | controller | - ' logger.go:42: 11:32:24 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421147 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421153 | controller | - ' logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421159 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421164 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421171 | controller | service' 2026-01-21 11:38:24.421177 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421183 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421189 | controller | - ' ]' 2026-01-21 11:38:24.421195 | controller | - ' logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421200 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421206 | controller | - ' logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421212 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421218 | controller | - ' logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421224 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421230 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421242 | controller | - ' logger.go:42: 11:32:25 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421248 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421254 | controller | - ' logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421260 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421266 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421271 | controller | service' 2026-01-21 11:38:24.421277 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421283 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421289 | controller | - ' ]' 2026-01-21 11:38:24.421295 | controller | - ' logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421301 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421307 | controller | - ' logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421313 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421319 | controller | - ' logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421325 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421331 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421337 | controller | - ' logger.go:42: 11:32:26 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421342 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421348 | controller | - ' logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421354 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421360 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421366 | controller | service' 2026-01-21 11:38:24.421372 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421378 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421384 | controller | - ' ]' 2026-01-21 11:38:24.421399 | controller | - ' logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421406 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421411 | controller | - ' logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421417 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421423 | controller | - ' logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421429 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421435 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421444 | controller | - ' logger.go:42: 11:32:27 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421450 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421456 | controller | - ' logger.go:42: 11:32:28 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421462 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421468 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421473 | controller | service' 2026-01-21 11:38:24.421479 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421485 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421491 | controller | - ' ]' 2026-01-21 11:38:24.421497 | controller | - ' logger.go:42: 11:32:28 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421503 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421509 | controller | - ' logger.go:42: 11:32:28 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421515 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421521 | controller | - ' logger.go:42: 11:32:29 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421535 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421542 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421550 | controller | - ' logger.go:42: 11:32:29 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421557 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421566 | controller | - ' logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421574 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421581 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421589 | controller | service' 2026-01-21 11:38:24.421596 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421603 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421611 | controller | - ' ]' 2026-01-21 11:38:24.421618 | controller | - ' logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421625 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421631 | controller | - ' logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421637 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421642 | controller | - ' logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421652 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421658 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421664 | controller | - ' logger.go:42: 11:32:30 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421670 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421676 | controller | - ' logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421693 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421700 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421706 | controller | service' 2026-01-21 11:38:24.421725 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421734 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421740 | controller | - ' ]' 2026-01-21 11:38:24.421746 | controller | - ' logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421752 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421758 | controller | - ' logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421763 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421769 | controller | - ' logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421775 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421781 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421787 | controller | - ' logger.go:42: 11:32:31 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421793 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421799 | controller | - ' logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421805 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421811 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421817 | controller | service' 2026-01-21 11:38:24.421823 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421828 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421834 | controller | - ' ]' 2026-01-21 11:38:24.421840 | controller | - ' logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421846 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421852 | controller | - ' logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421858 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421864 | controller | - ' logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421870 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421876 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421882 | controller | - ' logger.go:42: 11:32:32 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.421888 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.421894 | controller | - ' logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.421899 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.421905 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.421911 | controller | service' 2026-01-21 11:38:24.421917 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.421923 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.421929 | controller | - ' ]' 2026-01-21 11:38:24.421935 | controller | - ' logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.421941 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.421947 | controller | - ' logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.421966 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.421972 | controller | - ' logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.421983 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.421991 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.421999 | controller | - ' logger.go:42: 11:32:34 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422006 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422014 | controller | - ' logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422022 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422030 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422038 | controller | service' 2026-01-21 11:38:24.422045 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422052 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422059 | controller | - ' ]' 2026-01-21 11:38:24.422067 | controller | - ' logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422074 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422081 | controller | - ' logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422087 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422104 | controller | - ' logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.422110 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.422117 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.422122 | controller | - ' logger.go:42: 11:32:35 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422128 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422134 | controller | - ' logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422140 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422146 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422152 | controller | service' 2026-01-21 11:38:24.422158 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422164 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422170 | controller | - ' ]' 2026-01-21 11:38:24.422176 | controller | - ' logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422182 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422189 | controller | - ' logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422196 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422202 | controller | - ' logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | error: 2026-01-21 11:38:24.422207 | controller | error from server (NotFound): pods "watcher-kuttl-decision-engine-0" not found in 2026-01-21 11:38:24.422213 | controller | namespace "watcher-kuttl-default"' 2026-01-21 11:38:24.422219 | controller | - ' logger.go:42: 11:32:36 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422225 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422231 | controller | - ' logger.go:42: 11:32:37 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422237 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422246 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422251 | controller | service' 2026-01-21 11:38:24.422257 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422263 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422269 | controller | - ' ]' 2026-01-21 11:38:24.422275 | controller | - ' logger.go:42: 11:32:37 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422291 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422297 | controller | - ' logger.go:42: 11:32:37 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422303 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422309 | controller | - ' logger.go:42: 11:32:38 | watcher-cinder/1-deploy-watcher-no-cinder | Error 2026-01-21 11:38:24.422315 | controller | from server (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.422321 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.422327 | controller | - ' logger.go:42: 11:32:38 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422333 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422339 | controller | - ' logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422345 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422351 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422356 | controller | service' 2026-01-21 11:38:24.422362 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422368 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422374 | controller | - ' ]' 2026-01-21 11:38:24.422389 | controller | - ' logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422396 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422402 | controller | - ' logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422408 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422414 | controller | - ' logger.go:42: 11:32:39 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422420 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422425 | controller | - ' logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422431 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422437 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422443 | controller | service' 2026-01-21 11:38:24.422449 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422455 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422470 | controller | - ' ]' 2026-01-21 11:38:24.422476 | controller | - ' logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422482 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422488 | controller | - ' logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422493 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422499 | controller | - ' logger.go:42: 11:32:40 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422505 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422511 | controller | - ' logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422517 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422530 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422537 | controller | service' 2026-01-21 11:38:24.422543 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422551 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422559 | controller | - ' ]' 2026-01-21 11:38:24.422567 | controller | - ' logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422599 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422607 | controller | - ' logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422614 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422620 | controller | - ' logger.go:42: 11:32:41 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422627 | controller | 0 == 2 '']''' 2026-01-21 11:38:24.422634 | controller | - ' logger.go:42: 11:32:42 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422640 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422647 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422653 | controller | service' 2026-01-21 11:38:24.422660 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422668 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422675 | controller | - ' ]' 2026-01-21 11:38:24.422683 | controller | - ' logger.go:42: 11:32:42 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422691 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422698 | controller | - ' logger.go:42: 11:32:42 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422706 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422730 | controller | - ' logger.go:42: 11:32:43 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422743 | controller | 1 == 2 '']''' 2026-01-21 11:38:24.422751 | controller | - ' logger.go:42: 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422759 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422771 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422779 | controller | service' 2026-01-21 11:38:24.422786 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422793 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422800 | controller | - ' ]' 2026-01-21 11:38:24.422807 | controller | - ' logger.go:42: 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422814 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422821 | controller | - ' logger.go:42: 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422837 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422846 | controller | - ' logger.go:42: 11:32:44 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422854 | controller | 2 == 2 '']''' 2026-01-21 11:38:24.422862 | controller | - ' logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422869 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422876 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.422883 | controller | service' 2026-01-21 11:38:24.422907 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.422915 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.422923 | controller | - ' ]' 2026-01-21 11:38:24.422930 | controller | - ' logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.422937 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.422945 | controller | - ' logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.422952 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.422957 | controller | - ' logger.go:42: 11:32:45 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.422978 | controller | 2 == 2 '']''' 2026-01-21 11:38:24.422984 | controller | - ' logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.422990 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.422996 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.423002 | controller | service' 2026-01-21 11:38:24.423008 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.423014 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.423019 | controller | - ' ]' 2026-01-21 11:38:24.423035 | controller | - ' logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.423041 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423046 | controller | - ' logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.423052 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423058 | controller | - ' logger.go:42: 11:32:46 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.423064 | controller | 2 == 2 '']''' 2026-01-21 11:38:24.423070 | controller | - ' logger.go:42: 11:32:47 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.423076 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423082 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.423088 | controller | service' 2026-01-21 11:38:24.423142 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.423149 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.423155 | controller | - ' ]' 2026-01-21 11:38:24.423161 | controller | - ' logger.go:42: 11:32:47 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.423167 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423173 | controller | - ' logger.go:42: 11:32:47 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.423179 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423184 | controller | - ' logger.go:42: 11:32:48 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.423190 | controller | 2 == 2 '']''' 2026-01-21 11:38:24.423196 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | running 2026-01-21 11:38:24.423202 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423208 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.423214 | controller | service' 2026-01-21 11:38:24.423219 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 | grep -c ''Block 2026-01-21 11:38:24.423225 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.423232 | controller | - ' ]' 2026-01-21 11:38:24.423239 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | ++ oc 2026-01-21 11:38:24.423245 | controller | logs -n watcher-kuttl-default watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423251 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | ++ grep 2026-01-21 11:38:24.423258 | controller | -c ''Block storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423265 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | + ''['' 2026-01-21 11:38:24.423272 | controller | 2 == 2 '']''' 2026-01-21 11:38:24.423280 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/1-deploy-watcher-no-cinder | test step 2026-01-21 11:38:24.423296 | controller | completed 1-deploy-watcher-no-cinder' 2026-01-21 11:38:24.423308 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder | starting test step 2026-01-21 11:38:24.423349 | controller | 2-deploy-cinder' 2026-01-21 11:38:24.423358 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder | OpenStackControlPlane:watcher-kuttl-default/openstack 2026-01-21 11:38:24.423366 | controller | updated' 2026-01-21 11:38:24.423373 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423381 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423388 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423396 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423403 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423411 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423418 | controller | - ' ]' 2026-01-21 11:38:24.423426 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423434 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423457 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423466 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423474 | controller | - ' logger.go:42: 11:32:49 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423482 | controller | - ' logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423490 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423498 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423505 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423512 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423520 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423526 | controller | - ' ]' 2026-01-21 11:38:24.423532 | controller | - ' logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423538 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423545 | controller | - ' logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423552 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423560 | controller | - ' logger.go:42: 11:32:50 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423568 | controller | - ' logger.go:42: 11:32:51 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423576 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423583 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423591 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423599 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423607 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423614 | controller | - ' ]' 2026-01-21 11:38:24.423621 | controller | - ' logger.go:42: 11:32:51 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423627 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423633 | controller | - ' logger.go:42: 11:32:51 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423639 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423645 | controller | - ' logger.go:42: 11:32:52 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423651 | controller | - ' logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423657 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423663 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423673 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423679 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423685 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423691 | controller | - ' ]' 2026-01-21 11:38:24.423697 | controller | - ' logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423703 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423709 | controller | - ' logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423735 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423746 | controller | - ' logger.go:42: 11:32:53 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423753 | controller | - ' logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423759 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423764 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423770 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423776 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423782 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423788 | controller | - ' ]' 2026-01-21 11:38:24.423794 | controller | - ' logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423800 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423806 | controller | - ' logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423812 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423818 | controller | - ' logger.go:42: 11:32:54 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423824 | controller | - ' logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423829 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423835 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423841 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423847 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423853 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423859 | controller | - ' ]' 2026-01-21 11:38:24.423876 | controller | - ' logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423882 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423888 | controller | - ' logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423894 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423900 | controller | - ' logger.go:42: 11:32:55 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423906 | controller | - ' logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423912 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.423918 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.423924 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.423930 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.423936 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.423942 | controller | - ' ]' 2026-01-21 11:38:24.423948 | controller | - ' logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.423965 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.423971 | controller | - ' logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.423977 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.423983 | controller | - ' logger.go:42: 11:32:57 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.423989 | controller | - ' logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.423995 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424000 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424014 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424021 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424027 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424033 | controller | - ' ]' 2026-01-21 11:38:24.424039 | controller | - ' logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424044 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424050 | controller | - ' logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424056 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424062 | controller | - ' logger.go:42: 11:32:58 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424068 | controller | - ' logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424074 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424081 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424087 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424092 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424098 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424104 | controller | - ' ]' 2026-01-21 11:38:24.424110 | controller | - ' logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424116 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424122 | controller | - ' logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424128 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424134 | controller | - ' logger.go:42: 11:32:59 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424140 | controller | - ' logger.go:42: 11:33:00 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424146 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424151 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424157 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424163 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424169 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424175 | controller | - ' ]' 2026-01-21 11:38:24.424183 | controller | - ' logger.go:42: 11:33:00 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424189 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424195 | controller | - ' logger.go:42: 11:33:00 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424201 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424207 | controller | - ' logger.go:42: 11:33:01 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424216 | controller | - ' logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424225 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424231 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424237 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424251 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424257 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424263 | controller | - ' ]' 2026-01-21 11:38:24.424271 | controller | - ' logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424279 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424286 | controller | - ' logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424294 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424302 | controller | - ' logger.go:42: 11:33:02 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424310 | controller | - ' logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424317 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424325 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424332 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424339 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424346 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424354 | controller | - ' ]' 2026-01-21 11:38:24.424361 | controller | - ' logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424369 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424377 | controller | - ' logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424384 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424397 | controller | - ' logger.go:42: 11:33:03 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424405 | controller | - ' logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424412 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424419 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424426 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424434 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424440 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424446 | controller | - ' ]' 2026-01-21 11:38:24.424452 | controller | - ' logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424458 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424464 | controller | - ' logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424470 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424475 | controller | - ' logger.go:42: 11:33:04 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424481 | controller | - ' logger.go:42: 11:33:05 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424487 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424493 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424499 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424505 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424511 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424524 | controller | - ' ]' 2026-01-21 11:38:24.424530 | controller | - ' logger.go:42: 11:33:05 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424536 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424543 | controller | - ' logger.go:42: 11:33:05 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424550 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424558 | controller | - ' logger.go:42: 11:33:06 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424566 | controller | - ' logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424573 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424581 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424588 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424596 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424603 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424610 | controller | - ' ]' 2026-01-21 11:38:24.424617 | controller | - ' logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424624 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424631 | controller | - ' logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424637 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424643 | controller | - ' logger.go:42: 11:33:07 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424649 | controller | - ' logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424655 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424661 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424678 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424685 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424691 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424697 | controller | - ' ]' 2026-01-21 11:38:24.424703 | controller | - ' logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424709 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424731 | controller | - ' logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424737 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424743 | controller | - ' logger.go:42: 11:33:08 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424749 | controller | - ' logger.go:42: 11:33:09 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424755 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424765 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424771 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424777 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424783 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424789 | controller | - ' ]' 2026-01-21 11:38:24.424795 | controller | - ' logger.go:42: 11:33:09 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424801 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424807 | controller | - ' logger.go:42: 11:33:09 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424813 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424818 | controller | - ' logger.go:42: 11:33:10 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424828 | controller | - ' logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424834 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424840 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424846 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424852 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424857 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424863 | controller | - ' ]' 2026-01-21 11:38:24.424869 | controller | - ' logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424875 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424881 | controller | - ' logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424887 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424893 | controller | - ' logger.go:42: 11:33:11 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424899 | controller | - ' logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424904 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424910 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.424916 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.424922 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.424928 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.424934 | controller | - ' ]' 2026-01-21 11:38:24.424939 | controller | - ' logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.424945 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.424951 | controller | - ' logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.424957 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.424963 | controller | - ' logger.go:42: 11:33:12 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.424969 | controller | - ' logger.go:42: 11:33:13 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.424975 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.424985 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425003 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425018 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425026 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425034 | controller | - ' ]' 2026-01-21 11:38:24.425042 | controller | - ' logger.go:42: 11:33:13 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425049 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425055 | controller | - ' logger.go:42: 11:33:13 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.425061 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.425067 | controller | - ' logger.go:42: 11:33:14 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.425089 | controller | - ' logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.425096 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.425102 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425108 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425114 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425125 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425131 | controller | - ' ]' 2026-01-21 11:38:24.425138 | controller | - ' logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425144 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425150 | controller | - ' logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.425155 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.425161 | controller | - ' logger.go:42: 11:33:15 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.425167 | controller | - ' logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.425173 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.425179 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425185 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425190 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425196 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425202 | controller | - ' ]' 2026-01-21 11:38:24.425208 | controller | - ' logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425214 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425220 | controller | - ' logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.425226 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.425232 | controller | - ' logger.go:42: 11:33:16 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.425239 | controller | - ' logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.425246 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.425252 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425258 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425273 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425279 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425286 | controller | - ' ]' 2026-01-21 11:38:24.425293 | controller | - ' logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425392 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425399 | controller | - ' logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.425405 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.425411 | controller | - ' logger.go:42: 11:33:17 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.425417 | controller | - ' logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.425423 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.425429 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425435 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425441 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425447 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425453 | controller | - ' ]' 2026-01-21 11:38:24.425459 | controller | - ' logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425465 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425472 | controller | - ' logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.425495 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.425504 | controller | - ' logger.go:42: 11:33:19 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.425511 | controller | - ' logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.425519 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.425526 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425534 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425541 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425548 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425556 | controller | - ' ]' 2026-01-21 11:38:24.425564 | controller | - ' logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425571 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425579 | controller | - ' logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.425586 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.425604 | controller | - ' logger.go:42: 11:33:20 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.425613 | controller | - ' logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.425620 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.425628 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.425636 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.425643 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.425650 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.425729 | controller | - ' ]' 2026-01-21 11:38:24.425972 | controller | - ' logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.425988 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.425997 | controller | - ' logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426006 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426014 | controller | - ' logger.go:42: 11:33:21 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426022 | controller | - ' logger.go:42: 11:33:22 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426029 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426037 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426045 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426053 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426060 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426067 | controller | - ' ]' 2026-01-21 11:38:24.426074 | controller | - ' logger.go:42: 11:33:22 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426082 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426089 | controller | - ' logger.go:42: 11:33:22 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426095 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426101 | controller | - ' logger.go:42: 11:33:23 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426108 | controller | - ' logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426115 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426121 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426144 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426151 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426157 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426163 | controller | - ' ]' 2026-01-21 11:38:24.426168 | controller | - ' logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426174 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426180 | controller | - ' logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426186 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426192 | controller | - ' logger.go:42: 11:33:24 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426198 | controller | - ' logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426204 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426210 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426216 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426222 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426228 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426234 | controller | - ' ]' 2026-01-21 11:38:24.426240 | controller | - ' logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426247 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426259 | controller | - ' logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426265 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426271 | controller | - ' logger.go:42: 11:33:25 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426277 | controller | - ' logger.go:42: 11:33:26 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426283 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426289 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426295 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426301 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426307 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426312 | controller | - ' ]' 2026-01-21 11:38:24.426318 | controller | - ' logger.go:42: 11:33:26 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426324 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426374 | controller | - ' logger.go:42: 11:33:26 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426383 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426389 | controller | - ' logger.go:42: 11:33:27 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426396 | controller | - ' logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426402 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426408 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426414 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426421 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426428 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426434 | controller | - ' ]' 2026-01-21 11:38:24.426440 | controller | - ' logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426450 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426456 | controller | - ' logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426462 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426468 | controller | - ' logger.go:42: 11:33:28 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426474 | controller | - ' logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426480 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426486 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426509 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426520 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426528 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426537 | controller | - ' ]' 2026-01-21 11:38:24.426545 | controller | - ' logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426552 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426560 | controller | - ' logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426568 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426575 | controller | - ' logger.go:42: 11:33:29 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426582 | controller | - ' logger.go:42: 11:33:30 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426660 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426666 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426672 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426678 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426685 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426693 | controller | - ' ]' 2026-01-21 11:38:24.426699 | controller | - ' logger.go:42: 11:33:30 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426705 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426726 | controller | - ' logger.go:42: 11:33:30 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426737 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426744 | controller | - ' logger.go:42: 11:33:31 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426756 | controller | - ' logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426771 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426779 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426787 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426795 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426802 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426809 | controller | - ' ]' 2026-01-21 11:38:24.426817 | controller | - ' logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.426825 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.426833 | controller | - ' logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.426845 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.426859 | controller | - ' logger.go:42: 11:33:32 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.426867 | controller | - ' logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.426892 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.426901 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.426908 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.426952 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.426962 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.426979 | controller | - ' ]' 2026-01-21 11:38:24.427004 | controller | - ' logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427014 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427023 | controller | - ' logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427031 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427052 | controller | - ' logger.go:42: 11:33:33 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427061 | controller | - ' logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427068 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427076 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427084 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427093 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427101 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427108 | controller | - ' ]' 2026-01-21 11:38:24.427117 | controller | - ' logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427125 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427133 | controller | - ' logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427140 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427148 | controller | - ' logger.go:42: 11:33:34 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427156 | controller | - ' logger.go:42: 11:33:35 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427164 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427171 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427178 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427187 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427196 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427204 | controller | - ' ]' 2026-01-21 11:38:24.427212 | controller | - ' logger.go:42: 11:33:35 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427221 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427228 | controller | - ' logger.go:42: 11:33:35 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427236 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427243 | controller | - ' logger.go:42: 11:33:36 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427251 | controller | - ' logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427258 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427266 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427273 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427280 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427288 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427296 | controller | - ' ]' 2026-01-21 11:38:24.427342 | controller | - ' logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427351 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427359 | controller | - ' logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427366 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427374 | controller | - ' logger.go:42: 11:33:37 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427381 | controller | - ' logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427389 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427398 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427405 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427413 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427420 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427428 | controller | - ' ]' 2026-01-21 11:38:24.427436 | controller | - ' logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427443 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427451 | controller | - ' logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427458 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427466 | controller | - ' logger.go:42: 11:33:38 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427473 | controller | - ' logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427481 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427489 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427497 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427504 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427529 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427537 | controller | - ' ]' 2026-01-21 11:38:24.427545 | controller | - ' logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427553 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427561 | controller | - ' logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427570 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427585 | controller | - ' logger.go:42: 11:33:39 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427593 | controller | - ' logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427677 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427689 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427696 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427704 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427731 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427745 | controller | - ' ]' 2026-01-21 11:38:24.427753 | controller | - ' logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427807 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427817 | controller | - ' logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427825 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427832 | controller | - ' logger.go:42: 11:33:41 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427876 | controller | - ' logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427886 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427893 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427901 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.427909 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.427916 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.427924 | controller | - ' ]' 2026-01-21 11:38:24.427931 | controller | - ' logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.427939 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.427947 | controller | - ' logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.427955 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.427963 | controller | - ' logger.go:42: 11:33:42 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.427970 | controller | - ' logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.427978 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.427985 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.427993 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428001 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428008 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428016 | controller | - ' ]' 2026-01-21 11:38:24.428023 | controller | - ' logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428031 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428038 | controller | - ' logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428046 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428054 | controller | - ' logger.go:42: 11:33:43 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428061 | controller | - ' logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428068 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428076 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428171 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428180 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428186 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428192 | controller | - ' ]' 2026-01-21 11:38:24.428199 | controller | - ' logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428207 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428213 | controller | - ' logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428219 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428225 | controller | - ' logger.go:42: 11:33:44 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428231 | controller | - ' logger.go:42: 11:33:45 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428239 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428270 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428292 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428299 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428315 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428321 | controller | - ' ]' 2026-01-21 11:38:24.428328 | controller | - ' logger.go:42: 11:33:45 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428336 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428343 | controller | - ' logger.go:42: 11:33:45 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428350 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428356 | controller | - ' logger.go:42: 11:33:46 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428362 | controller | - ' logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428368 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428374 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428380 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428386 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428392 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428398 | controller | - ' ]' 2026-01-21 11:38:24.428404 | controller | - ' logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428410 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428416 | controller | - ' logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428422 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428428 | controller | - ' logger.go:42: 11:33:47 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428434 | controller | - ' logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428440 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428446 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428452 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428458 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428464 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428471 | controller | - ' ]' 2026-01-21 11:38:24.428478 | controller | - ' logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428485 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428491 | controller | - ' logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428497 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428503 | controller | - ' logger.go:42: 11:33:48 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428509 | controller | - ' logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428515 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428535 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428542 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428548 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428554 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428560 | controller | - ' ]' 2026-01-21 11:38:24.428566 | controller | - ' logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428572 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428578 | controller | - ' logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428592 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428598 | controller | - ' logger.go:42: 11:33:49 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428604 | controller | - ' logger.go:42: 11:33:50 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428612 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428619 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428625 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428631 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428637 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428643 | controller | - ' ]' 2026-01-21 11:38:24.428649 | controller | - ' logger.go:42: 11:33:50 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428655 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428661 | controller | - ' logger.go:42: 11:33:50 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428667 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428673 | controller | - ' logger.go:42: 11:33:51 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428679 | controller | - ' logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428695 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428702 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428708 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428741 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428747 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428754 | controller | - ' ]' 2026-01-21 11:38:24.428761 | controller | - ' logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428767 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428773 | controller | - ' logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428779 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428785 | controller | - ' logger.go:42: 11:33:52 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428791 | controller | - ' logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428797 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428803 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428809 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428814 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428820 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428826 | controller | - ' ]' 2026-01-21 11:38:24.428832 | controller | - ' logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428838 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428844 | controller | - ' logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428850 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428856 | controller | - ' logger.go:42: 11:33:53 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428862 | controller | - ' logger.go:42: 11:33:54 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428868 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428874 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428883 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428891 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428898 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428905 | controller | - ' ]' 2026-01-21 11:38:24.428911 | controller | - ' logger.go:42: 11:33:54 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428917 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428923 | controller | - ' logger.go:42: 11:33:54 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.428929 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.428935 | controller | - ' logger.go:42: 11:33:55 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.428941 | controller | - ' logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.428946 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.428952 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.428958 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.428964 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.428970 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.428976 | controller | - ' ]' 2026-01-21 11:38:24.428982 | controller | - ' logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.428988 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.428994 | controller | - ' logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429000 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429006 | controller | - ' logger.go:42: 11:33:56 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429012 | controller | - ' logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429018 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429025 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429033 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429050 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429057 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429063 | controller | - ' ]' 2026-01-21 11:38:24.429069 | controller | - ' logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429075 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429081 | controller | - ' logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429087 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429105 | controller | - ' logger.go:42: 11:33:57 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429112 | controller | - ' logger.go:42: 11:33:58 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429118 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429124 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429130 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429136 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429142 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429148 | controller | - ' ]' 2026-01-21 11:38:24.429154 | controller | - ' logger.go:42: 11:33:58 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429161 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429172 | controller | - ' logger.go:42: 11:33:58 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429228 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429237 | controller | - ' logger.go:42: 11:33:59 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429243 | controller | - ' logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429249 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429255 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429264 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429269 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429275 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429281 | controller | - ' ]' 2026-01-21 11:38:24.429287 | controller | - ' logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429294 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429301 | controller | - ' logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429308 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429314 | controller | - ' logger.go:42: 11:34:00 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429320 | controller | - ' logger.go:42: 11:34:01 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429326 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429332 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429338 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429344 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429350 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429356 | controller | - ' ]' 2026-01-21 11:38:24.429381 | controller | - ' logger.go:42: 11:34:01 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429388 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429394 | controller | - ' logger.go:42: 11:34:01 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429400 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429406 | controller | - ' logger.go:42: 11:34:02 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429412 | controller | - ' logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429418 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429424 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429430 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429437 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429444 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429450 | controller | - ' ]' 2026-01-21 11:38:24.429456 | controller | - ' logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429462 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429468 | controller | - ' logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429474 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429480 | controller | - ' logger.go:42: 11:34:03 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429486 | controller | - ' logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429491 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429515 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429522 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429528 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429534 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429540 | controller | - ' ]' 2026-01-21 11:38:24.429546 | controller | - ' logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429552 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429558 | controller | - ' logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429575 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429583 | controller | - ' logger.go:42: 11:34:04 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429590 | controller | - ' logger.go:42: 11:34:05 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429596 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429602 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429608 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429614 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429619 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429625 | controller | - ' ]' 2026-01-21 11:38:24.429631 | controller | - ' logger.go:42: 11:34:05 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429637 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429653 | controller | - ' logger.go:42: 11:34:05 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429659 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429665 | controller | - ' logger.go:42: 11:34:06 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429671 | controller | - ' logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429677 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429683 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429689 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429694 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429700 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429706 | controller | - ' ]' 2026-01-21 11:38:24.429733 | controller | - ' logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429740 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429746 | controller | - ' logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429753 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429766 | controller | - ' logger.go:42: 11:34:07 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429780 | controller | - ' logger.go:42: 11:34:08 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429788 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429794 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429800 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429806 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429812 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429818 | controller | - ' ]' 2026-01-21 11:38:24.429851 | controller | - ' logger.go:42: 11:34:08 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429860 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429868 | controller | - ' logger.go:42: 11:34:08 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429874 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429880 | controller | - ' logger.go:42: 11:34:09 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429886 | controller | - ' logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429892 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429898 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429904 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429910 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.429916 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.429922 | controller | - ' ]' 2026-01-21 11:38:24.429928 | controller | - ' logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.429933 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.429940 | controller | - ' logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.429946 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.429952 | controller | - ' logger.go:42: 11:34:10 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.429958 | controller | - ' logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.429964 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.429970 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.429978 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.429995 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430005 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430012 | controller | - ' ]' 2026-01-21 11:38:24.430020 | controller | - ' logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430042 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430049 | controller | - ' logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430056 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430062 | controller | - ' logger.go:42: 11:34:11 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430067 | controller | - ' logger.go:42: 11:34:12 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430073 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430079 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430086 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430092 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430098 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430104 | controller | - ' ]' 2026-01-21 11:38:24.430110 | controller | - ' logger.go:42: 11:34:12 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430116 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430122 | controller | - ' logger.go:42: 11:34:12 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430128 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430135 | controller | - ' logger.go:42: 11:34:13 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430169 | controller | - ' logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430178 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430184 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430190 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430196 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430202 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430207 | controller | - ' ]' 2026-01-21 11:38:24.430213 | controller | - ' logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430219 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430225 | controller | - ' logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430232 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430258 | controller | - ' logger.go:42: 11:34:14 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430268 | controller | - ' logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430276 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430284 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430291 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430297 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430303 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430309 | controller | - ' ]' 2026-01-21 11:38:24.430315 | controller | - ' logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430321 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430327 | controller | - ' logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430333 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430339 | controller | - ' logger.go:42: 11:34:15 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430345 | controller | - ' logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430351 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430357 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430366 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430372 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430378 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430383 | controller | - ' ]' 2026-01-21 11:38:24.430389 | controller | - ' logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430395 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430401 | controller | - ' logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430408 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430439 | controller | - ' logger.go:42: 11:34:17 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430445 | controller | - ' logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430451 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430457 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430463 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430469 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430489 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430497 | controller | - ' ]' 2026-01-21 11:38:24.430503 | controller | - ' logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430510 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430516 | controller | - ' logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430522 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430528 | controller | - ' logger.go:42: 11:34:18 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430533 | controller | - ' logger.go:42: 11:34:19 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430539 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430568 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430575 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430581 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430631 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430637 | controller | - ' ]' 2026-01-21 11:38:24.430643 | controller | - ' logger.go:42: 11:34:19 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430650 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430656 | controller | - ' logger.go:42: 11:34:19 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430662 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430668 | controller | - ' logger.go:42: 11:34:20 | watcher-cinder/2-deploy-cinder | + ''['' 2 == 0 '']''' 2026-01-21 11:38:24.430673 | controller | - ' logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430679 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430686 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430693 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430700 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430706 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430726 | controller | - ' ]' 2026-01-21 11:38:24.430737 | controller | - ' logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430743 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430749 | controller | - ' logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430755 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430760 | controller | - ' logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder | Error from server 2026-01-21 11:38:24.430766 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.430772 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.430778 | controller | - ' logger.go:42: 11:34:21 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.430784 | controller | - ' logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.430891 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.430908 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.430914 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.430920 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.430926 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.430932 | controller | - ' ]' 2026-01-21 11:38:24.430937 | controller | - ' logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.430964 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.430973 | controller | - ' logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.430981 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.430994 | controller | - ' logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder | Error from server 2026-01-21 11:38:24.431003 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.431016 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.431030 | controller | - ' logger.go:42: 11:34:22 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431038 | controller | - ' logger.go:42: 11:34:23 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431046 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431053 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431061 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431069 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431077 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431084 | controller | - ' ]' 2026-01-21 11:38:24.431092 | controller | - ' logger.go:42: 11:34:23 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431099 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431124 | controller | - ' logger.go:42: 11:34:23 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431133 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431141 | controller | - ' logger.go:42: 11:34:24 | watcher-cinder/2-deploy-cinder | Error from server 2026-01-21 11:38:24.431149 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.431158 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.431166 | controller | - ' logger.go:42: 11:34:24 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431174 | controller | - ' logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431182 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431191 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431199 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431206 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431214 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431221 | controller | - ' ]' 2026-01-21 11:38:24.431228 | controller | - ' logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431235 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431242 | controller | - ' logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431250 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431257 | controller | - ' logger.go:42: 11:34:25 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431265 | controller | - ' logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431273 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431281 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431289 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431297 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431304 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431326 | controller | - ' ]' 2026-01-21 11:38:24.431333 | controller | - ' logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431339 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431345 | controller | - ' logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431351 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431357 | controller | - ' logger.go:42: 11:34:26 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431363 | controller | - ' logger.go:42: 11:34:27 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431370 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431377 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431384 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431390 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431396 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431401 | controller | - ' ]' 2026-01-21 11:38:24.431407 | controller | - ' logger.go:42: 11:34:27 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431413 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431419 | controller | - ' logger.go:42: 11:34:27 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431425 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431455 | controller | - ' logger.go:42: 11:34:28 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431464 | controller | - ' logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431470 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431475 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431481 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431487 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431493 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431499 | controller | - ' ]' 2026-01-21 11:38:24.431507 | controller | - ' logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431514 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431520 | controller | - ' logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431526 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431531 | controller | - ' logger.go:42: 11:34:29 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431537 | controller | - ' logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431543 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431549 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431555 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431573 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431579 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431585 | controller | - ' ]' 2026-01-21 11:38:24.431591 | controller | - ' logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431597 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431607 | controller | - ' logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431613 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431623 | controller | - ' logger.go:42: 11:34:30 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431629 | controller | - ' logger.go:42: 11:34:31 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431636 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431643 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431650 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431656 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431662 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431668 | controller | - ' ]' 2026-01-21 11:38:24.431673 | controller | - ' logger.go:42: 11:34:31 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431679 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431685 | controller | - ' logger.go:42: 11:34:31 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431691 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431697 | controller | - ' logger.go:42: 11:34:32 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431703 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder | running command: 2026-01-21 11:38:24.431731 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431743 | controller | - ' # check that the decision detects that there is a cinder service and' 2026-01-21 11:38:24.431760 | controller | - ' # does not log that storage collector is skipped' 2026-01-21 11:38:24.431767 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431774 | controller | storage service is not enabled, skipping storage collector'')" == 0 ]' 2026-01-21 11:38:24.431781 | controller | - ' ]' 2026-01-21 11:38:24.431788 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431794 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431799 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431805 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431811 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.431817 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/2-deploy-cinder | test step completed 2026-01-21 11:38:24.431823 | controller | 2-deploy-cinder' 2026-01-21 11:38:24.431829 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder | starting test step 2026-01-21 11:38:24.431835 | controller | 3-remove-cinder' 2026-01-21 11:38:24.431840 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder | OpenStackControlPlane:watcher-kuttl-default/openstack 2026-01-21 11:38:24.431847 | controller | updated' 2026-01-21 11:38:24.431853 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.431859 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431864 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.431870 | controller | service' 2026-01-21 11:38:24.431876 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431882 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.431888 | controller | - ' ]' 2026-01-21 11:38:24.431894 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.431900 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.431906 | controller | - ' logger.go:42: 11:34:33 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.431912 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.431919 | controller | - ' logger.go:42: 11:34:34 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.431948 | controller | - ' logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.431955 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.431961 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.431968 | controller | service' 2026-01-21 11:38:24.431977 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.431994 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432003 | controller | - ' ]' 2026-01-21 11:38:24.432010 | controller | - ' logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432018 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432025 | controller | - ' logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432032 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432057 | controller | - ' logger.go:42: 11:34:35 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432066 | controller | - ' logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432073 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432079 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432085 | controller | service' 2026-01-21 11:38:24.432091 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432097 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432103 | controller | - ' ]' 2026-01-21 11:38:24.432109 | controller | - ' logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432114 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432120 | controller | - ' logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432126 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432132 | controller | - ' logger.go:42: 11:34:36 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432138 | controller | - ' logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432144 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432150 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432155 | controller | service' 2026-01-21 11:38:24.432166 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432172 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432178 | controller | - ' ]' 2026-01-21 11:38:24.432185 | controller | - ' logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432192 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432198 | controller | - ' logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432204 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432210 | controller | - ' logger.go:42: 11:34:37 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432216 | controller | - ' logger.go:42: 11:34:38 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432222 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432227 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432235 | controller | service' 2026-01-21 11:38:24.432242 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432248 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432284 | controller | - ' ]' 2026-01-21 11:38:24.432292 | controller | - ' logger.go:42: 11:34:38 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432298 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432304 | controller | - ' logger.go:42: 11:34:38 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432310 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432316 | controller | - ' logger.go:42: 11:34:39 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432323 | controller | - ' logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432330 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432336 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432342 | controller | service' 2026-01-21 11:38:24.432348 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432354 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432360 | controller | - ' ]' 2026-01-21 11:38:24.432366 | controller | - ' logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432372 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432378 | controller | - ' logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432384 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432389 | controller | - ' logger.go:42: 11:34:40 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432395 | controller | - ' logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432401 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432407 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432413 | controller | service' 2026-01-21 11:38:24.432419 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432425 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432431 | controller | - ' ]' 2026-01-21 11:38:24.432437 | controller | - ' logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432443 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432449 | controller | - ' logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432455 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432461 | controller | - ' logger.go:42: 11:34:41 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432479 | controller | - ' logger.go:42: 11:34:42 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432486 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432492 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432498 | controller | service' 2026-01-21 11:38:24.432504 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432510 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432516 | controller | - ' ]' 2026-01-21 11:38:24.432522 | controller | - ' logger.go:42: 11:34:43 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432528 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432534 | controller | - ' logger.go:42: 11:34:43 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432540 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432546 | controller | - ' logger.go:42: 11:34:43 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432551 | controller | - ' logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432561 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432567 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432573 | controller | service' 2026-01-21 11:38:24.432579 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432584 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432590 | controller | - ' ]' 2026-01-21 11:38:24.432597 | controller | - ' logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432604 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432610 | controller | - ' logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432616 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432622 | controller | - ' logger.go:42: 11:34:44 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432628 | controller | - ' logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432634 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432640 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432646 | controller | service' 2026-01-21 11:38:24.432651 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432657 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432663 | controller | - ' ]' 2026-01-21 11:38:24.432669 | controller | - ' logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432675 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432702 | controller | - ' logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432726 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432737 | controller | - ' logger.go:42: 11:34:45 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432743 | controller | - ' logger.go:42: 11:34:46 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432749 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432755 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432761 | controller | service' 2026-01-21 11:38:24.432767 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432773 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432779 | controller | - ' ]' 2026-01-21 11:38:24.432785 | controller | - ' logger.go:42: 11:34:46 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432791 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432797 | controller | - ' logger.go:42: 11:34:46 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432803 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432809 | controller | - ' logger.go:42: 11:34:47 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432815 | controller | - ' logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432820 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432826 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432832 | controller | service' 2026-01-21 11:38:24.432840 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432846 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432852 | controller | - ' ]' 2026-01-21 11:38:24.432858 | controller | - ' logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.432879 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.432885 | controller | - ' logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.432891 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.432897 | controller | - ' logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.432914 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.432921 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.432927 | controller | - ' logger.go:42: 11:34:48 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.432933 | controller | - ' logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.432939 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.432968 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.432976 | controller | service' 2026-01-21 11:38:24.432984 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.432991 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.432999 | controller | - ' ]' 2026-01-21 11:38:24.433007 | controller | - ' logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433019 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433027 | controller | - ' logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433034 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433041 | controller | - ' logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.433048 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.433056 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.433063 | controller | - ' logger.go:42: 11:34:49 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433068 | controller | - ' logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433074 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433080 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433086 | controller | service' 2026-01-21 11:38:24.433092 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433098 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433104 | controller | - ' ]' 2026-01-21 11:38:24.433110 | controller | - ' logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433116 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433124 | controller | - ' logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433131 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433138 | controller | - ' logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.433144 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.433149 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.433155 | controller | - ' logger.go:42: 11:34:50 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433161 | controller | - ' logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433167 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433173 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433179 | controller | service' 2026-01-21 11:38:24.433185 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433195 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433201 | controller | - ' ]' 2026-01-21 11:38:24.433207 | controller | - ' logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433213 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433219 | controller | - ' logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433224 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433230 | controller | - ' logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.433236 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.433242 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.433248 | controller | - ' logger.go:42: 11:34:52 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433257 | controller | - ' logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433265 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433272 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433278 | controller | service' 2026-01-21 11:38:24.433284 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433290 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433296 | controller | - ' ]' 2026-01-21 11:38:24.433302 | controller | - ' logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433308 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433314 | controller | - ' logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433320 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433326 | controller | - ' logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder | Error from server 2026-01-21 11:38:24.433344 | controller | (BadRequest): container "watcher-decision-engine" in pod "watcher-kuttl-decision-engine-0" 2026-01-21 11:38:24.433350 | controller | is waiting to start: ContainerCreating' 2026-01-21 11:38:24.433357 | controller | - ' logger.go:42: 11:34:53 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433363 | controller | - ' logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433369 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433375 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433381 | controller | service' 2026-01-21 11:38:24.433387 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433393 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433399 | controller | - ' ]' 2026-01-21 11:38:24.433405 | controller | - ' logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433412 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433432 | controller | - ' logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433438 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433444 | controller | - ' logger.go:42: 11:34:54 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433450 | controller | - ' logger.go:42: 11:34:55 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433456 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433462 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433468 | controller | service' 2026-01-21 11:38:24.433474 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433483 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433489 | controller | - ' ]' 2026-01-21 11:38:24.433495 | controller | - ' logger.go:42: 11:34:55 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433501 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433507 | controller | - ' logger.go:42: 11:34:55 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433513 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433519 | controller | - ' logger.go:42: 11:34:56 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433525 | controller | - ' logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433530 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433536 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433542 | controller | service' 2026-01-21 11:38:24.433549 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433557 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433564 | controller | - ' ]' 2026-01-21 11:38:24.433570 | controller | - ' logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433576 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433582 | controller | - ' logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433588 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433594 | controller | - ' logger.go:42: 11:34:57 | watcher-cinder/3-remove-cinder | + ''['' 0 == 2 '']''' 2026-01-21 11:38:24.433600 | controller | - ' logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433606 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433612 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433618 | controller | service' 2026-01-21 11:38:24.433624 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433630 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433635 | controller | - ' ]' 2026-01-21 11:38:24.433643 | controller | - ' logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433655 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433669 | controller | - ' logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433677 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433684 | controller | - ' logger.go:42: 11:34:58 | watcher-cinder/3-remove-cinder | + ''['' 1 == 2 '']''' 2026-01-21 11:38:24.433692 | controller | - ' logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433699 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433707 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433746 | controller | service' 2026-01-21 11:38:24.433756 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433763 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433770 | controller | - ' ]' 2026-01-21 11:38:24.433778 | controller | - ' logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433786 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433793 | controller | - ' logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433801 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433823 | controller | - ' logger.go:42: 11:34:59 | watcher-cinder/3-remove-cinder | + ''['' 1 == 2 '']''' 2026-01-21 11:38:24.433856 | controller | - ' logger.go:42: 11:35:00 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433865 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433872 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433879 | controller | service' 2026-01-21 11:38:24.433886 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433893 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.433900 | controller | - ' ]' 2026-01-21 11:38:24.433908 | controller | - ' logger.go:42: 11:35:00 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.433916 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.433929 | controller | - ' logger.go:42: 11:35:00 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.433942 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.433949 | controller | - ' logger.go:42: 11:35:01 | watcher-cinder/3-remove-cinder | + ''['' 1 == 2 '']''' 2026-01-21 11:38:24.433957 | controller | - ' logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.433964 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.433972 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.433979 | controller | service' 2026-01-21 11:38:24.433987 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.433994 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.434002 | controller | - ' ]' 2026-01-21 11:38:24.434009 | controller | - ' logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.434017 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.434024 | controller | - ' logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.434031 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.434038 | controller | - ' logger.go:42: 11:35:02 | watcher-cinder/3-remove-cinder | + ''['' 1 == 2 '']''' 2026-01-21 11:38:24.434045 | controller | - ' logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.434053 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.434060 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.434066 | controller | service' 2026-01-21 11:38:24.434072 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.434078 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.434084 | controller | - ' ]' 2026-01-21 11:38:24.434090 | controller | - ' logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.434096 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.434103 | controller | - ' logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.434109 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.434115 | controller | - ' logger.go:42: 11:35:03 | watcher-cinder/3-remove-cinder | + ''['' 1 == 2 '']''' 2026-01-21 11:38:24.434121 | controller | - ' logger.go:42: 11:35:04 | watcher-cinder/3-remove-cinder | running command: 2026-01-21 11:38:24.434127 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.434133 | controller | - ' # check that the decision engine correctly detects that there is no cinder 2026-01-21 11:38:24.434139 | controller | service' 2026-01-21 11:38:24.434145 | controller | - ' [ "$(oc logs -n $NAMESPACE watcher-kuttl-decision-engine-0 |grep -c ''Block 2026-01-21 11:38:24.434151 | controller | storage service is not enabled, skipping storage collector'')" == 2 ]' 2026-01-21 11:38:24.434174 | controller | - ' ]' 2026-01-21 11:38:24.434181 | controller | - ' logger.go:42: 11:35:04 | watcher-cinder/3-remove-cinder | ++ oc logs -n watcher-kuttl-default 2026-01-21 11:38:24.434189 | controller | watcher-kuttl-decision-engine-0' 2026-01-21 11:38:24.434195 | controller | - ' logger.go:42: 11:35:04 | watcher-cinder/3-remove-cinder | ++ grep -c ''Block 2026-01-21 11:38:24.434200 | controller | storage service is not enabled, skipping storage collector''' 2026-01-21 11:38:24.434206 | controller | - ' logger.go:42: 11:35:05 | watcher-cinder/3-remove-cinder | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.434212 | controller | - ' logger.go:42: 11:35:05 | watcher-cinder/3-remove-cinder | test step completed 2026-01-21 11:38:24.434218 | controller | 3-remove-cinder' 2026-01-21 11:38:24.434224 | controller | - ' logger.go:42: 11:35:05 | watcher-cinder/4-cleanup-watcher | starting test step 2026-01-21 11:38:24.434231 | controller | 4-cleanup-watcher' 2026-01-21 11:38:24.434237 | controller | - ' logger.go:42: 11:35:16 | watcher-cinder/4-cleanup-watcher | test step completed 2026-01-21 11:38:24.434243 | controller | 4-cleanup-watcher' 2026-01-21 11:38:24.434249 | controller | - ' logger.go:42: 11:35:16 | watcher-cinder | skipping kubernetes event logging' 2026-01-21 11:38:24.434255 | controller | - === CONT kuttl/harness/watcher-api-scaling 2026-01-21 11:38:24.434261 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling | Skipping creation of user-supplied 2026-01-21 11:38:24.434267 | controller | namespace: watcher-kuttl-default' 2026-01-21 11:38:24.434274 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/0-cleanup-watcher | starting test 2026-01-21 11:38:24.434288 | controller | step 0-cleanup-watcher' 2026-01-21 11:38:24.434318 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/0-cleanup-watcher | test step 2026-01-21 11:38:24.434329 | controller | completed 0-cleanup-watcher' 2026-01-21 11:38:24.434337 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults | starting 2026-01-21 11:38:24.434344 | controller | test step 1-deploy-with-defaults' 2026-01-21 11:38:24.434352 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.434359 | controller | created' 2026-01-21 11:38:24.434366 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults | running 2026-01-21 11:38:24.434373 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.434380 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.434388 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.434396 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.434439 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.434450 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.434458 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.434465 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.434473 | controller | ]' 2026-01-21 11:38:24.434481 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.434489 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.434497 | controller | == "" ]; then' 2026-01-21 11:38:24.434505 | controller | - ' exit 0' 2026-01-21 11:38:24.434513 | controller | - ' fi' 2026-01-21 11:38:24.434520 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.434527 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.434535 | controller | - ' counter=0' 2026-01-21 11:38:24.434542 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.434550 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.434564 | controller | - ' echo ${i}' 2026-01-21 11:38:24.434572 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.434579 | controller | - ' fi' 2026-01-21 11:38:24.434604 | controller | - ' done' 2026-01-21 11:38:24.434613 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.434620 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.434628 | controller | - ' exit 1' 2026-01-21 11:38:24.434636 | controller | - ' else' 2026-01-21 11:38:24.434657 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.434665 | controller | - ' fi' 2026-01-21 11:38:24.434672 | controller | - ' ]' 2026-01-21 11:38:24.434679 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults | + oc 2026-01-21 11:38:24.434686 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.434693 | controller | -c Name -c Type' 2026-01-21 11:38:24.434700 | controller | - ' logger.go:42: 11:35:16 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.434707 | controller | -c ''^watcher''' 2026-01-21 11:38:24.434738 | controller | - ' logger.go:42: 11:35:21 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.434747 | controller | 0 == 1 '']''' 2026-01-21 11:38:24.434754 | controller | - ' logger.go:42: 11:35:23 | watcher-api-scaling/1-deploy-with-defaults | running 2026-01-21 11:38:24.434762 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.434775 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.434788 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.434796 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.434803 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.434811 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.434818 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.434826 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.434834 | controller | ]' 2026-01-21 11:38:24.434841 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.434848 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.434855 | controller | == "" ]; then' 2026-01-21 11:38:24.434863 | controller | - ' exit 0' 2026-01-21 11:38:24.434870 | controller | - ' fi' 2026-01-21 11:38:24.434878 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.434885 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.434892 | controller | - ' counter=0' 2026-01-21 11:38:24.434900 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.434907 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.434914 | controller | - ' echo ${i}' 2026-01-21 11:38:24.434921 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.434929 | controller | - ' fi' 2026-01-21 11:38:24.434936 | controller | - ' done' 2026-01-21 11:38:24.434943 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.434951 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.434958 | controller | - ' exit 1' 2026-01-21 11:38:24.434965 | controller | - ' else' 2026-01-21 11:38:24.434972 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.434980 | controller | - ' fi' 2026-01-21 11:38:24.434987 | controller | - ' ]' 2026-01-21 11:38:24.435013 | controller | - ' logger.go:42: 11:35:23 | watcher-api-scaling/1-deploy-with-defaults | + oc 2026-01-21 11:38:24.435023 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.435063 | controller | -c Name -c Type' 2026-01-21 11:38:24.435071 | controller | - ' logger.go:42: 11:35:23 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.435078 | controller | -c ''^watcher''' 2026-01-21 11:38:24.435086 | controller | - ' logger.go:42: 11:35:26 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435093 | controller | 0 == 1 '']''' 2026-01-21 11:38:24.435099 | controller | - ' logger.go:42: 11:35:27 | watcher-api-scaling/1-deploy-with-defaults | running 2026-01-21 11:38:24.435105 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.435111 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.435117 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.435122 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.435128 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.435134 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.435140 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.435146 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.435152 | controller | ]' 2026-01-21 11:38:24.435158 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.435164 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.435170 | controller | == "" ]; then' 2026-01-21 11:38:24.435175 | controller | - ' exit 0' 2026-01-21 11:38:24.435181 | controller | - ' fi' 2026-01-21 11:38:24.435187 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.435193 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.435199 | controller | - ' counter=0' 2026-01-21 11:38:24.435205 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.435211 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.435217 | controller | - ' echo ${i}' 2026-01-21 11:38:24.435223 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.435231 | controller | - ' fi' 2026-01-21 11:38:24.435239 | controller | - ' done' 2026-01-21 11:38:24.435245 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.435251 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.435257 | controller | - ' exit 1' 2026-01-21 11:38:24.435264 | controller | - ' else' 2026-01-21 11:38:24.435271 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.435277 | controller | - ' fi' 2026-01-21 11:38:24.435285 | controller | - ' ]' 2026-01-21 11:38:24.435291 | controller | - ' logger.go:42: 11:35:27 | watcher-api-scaling/1-deploy-with-defaults | + oc 2026-01-21 11:38:24.435298 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.435305 | controller | -c Name -c Type' 2026-01-21 11:38:24.435312 | controller | - ' logger.go:42: 11:35:27 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.435318 | controller | -c ''^watcher''' 2026-01-21 11:38:24.435324 | controller | - ' logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435330 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.435336 | controller | - ' logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435342 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.435348 | controller | -c Name -c Type -c ID' 2026-01-21 11:38:24.435353 | controller | - ' logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.435359 | controller | watcher' 2026-01-21 11:38:24.435388 | controller | - ' logger.go:42: 11:35:30 | watcher-api-scaling/1-deploy-with-defaults | ++ awk 2026-01-21 11:38:24.435395 | controller | ''{print $1}''' 2026-01-21 11:38:24.435401 | controller | - ' logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c' 2026-01-21 11:38:24.435407 | controller | - ' logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435413 | controller | get -n watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.435433 | controller | - ' logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435440 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c '']''' 2026-01-21 11:38:24.435464 | controller | - ' logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435472 | controller | get -n watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.435478 | controller | - ' logger.go:42: 11:35:33 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435484 | controller | -n '''' '']''' 2026-01-21 11:38:24.435490 | controller | - ' logger.go:42: 11:35:35 | watcher-api-scaling/1-deploy-with-defaults | running 2026-01-21 11:38:24.435496 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.435502 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.435508 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.435514 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.435530 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.435537 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.435544 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.435551 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.435557 | controller | ]' 2026-01-21 11:38:24.435563 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.435569 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.435575 | controller | == "" ]; then' 2026-01-21 11:38:24.435581 | controller | - ' exit 0' 2026-01-21 11:38:24.435587 | controller | - ' fi' 2026-01-21 11:38:24.435593 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.435599 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.435605 | controller | - ' counter=0' 2026-01-21 11:38:24.435610 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.435617 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.435624 | controller | - ' echo ${i}' 2026-01-21 11:38:24.435630 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.435637 | controller | - ' fi' 2026-01-21 11:38:24.435645 | controller | - ' done' 2026-01-21 11:38:24.435652 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.435660 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.435667 | controller | - ' exit 1' 2026-01-21 11:38:24.435674 | controller | - ' else' 2026-01-21 11:38:24.435695 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.435703 | controller | - ' fi' 2026-01-21 11:38:24.435752 | controller | - ' ]' 2026-01-21 11:38:24.435769 | controller | - ' logger.go:42: 11:35:35 | watcher-api-scaling/1-deploy-with-defaults | + oc 2026-01-21 11:38:24.435777 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.435785 | controller | -c Name -c Type' 2026-01-21 11:38:24.435793 | controller | - ' logger.go:42: 11:35:35 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.435820 | controller | -c ''^watcher''' 2026-01-21 11:38:24.435828 | controller | - ' logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435836 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.435843 | controller | - ' logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435850 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.435858 | controller | -c Name -c Type -c ID' 2026-01-21 11:38:24.435865 | controller | - ' logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults | ++ awk 2026-01-21 11:38:24.435873 | controller | ''{print $1}''' 2026-01-21 11:38:24.435881 | controller | - ' logger.go:42: 11:35:38 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.435889 | controller | watcher' 2026-01-21 11:38:24.435898 | controller | - ' logger.go:42: 11:35:41 | watcher-api-scaling/1-deploy-with-defaults | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c' 2026-01-21 11:38:24.435905 | controller | - ' logger.go:42: 11:35:41 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435913 | controller | get -n watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.435921 | controller | - ' logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435928 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c '']''' 2026-01-21 11:38:24.435950 | controller | - ' logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435958 | controller | get -n watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.435966 | controller | - ' logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.435974 | controller | -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.435982 | controller | '']''' 2026-01-21 11:38:24.435990 | controller | - ' logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.435998 | controller | get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.436006 | controller | - ' logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436014 | controller | '''' == '''' '']''' 2026-01-21 11:38:24.436021 | controller | - ' logger.go:42: 11:35:42 | watcher-api-scaling/1-deploy-with-defaults | + exit 2026-01-21 11:38:24.436029 | controller | 0' 2026-01-21 11:38:24.436037 | controller | - ' logger.go:42: 11:35:43 | watcher-api-scaling/1-deploy-with-defaults | running 2026-01-21 11:38:24.436045 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.436052 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.436060 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.436068 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.436076 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.436084 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.436092 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.436115 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.436123 | controller | ]' 2026-01-21 11:38:24.436131 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.436139 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.436146 | controller | == "" ]; then' 2026-01-21 11:38:24.436154 | controller | - ' exit 0' 2026-01-21 11:38:24.436161 | controller | - ' fi' 2026-01-21 11:38:24.436168 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.436208 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.436218 | controller | - ' counter=0' 2026-01-21 11:38:24.436226 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.436239 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.436247 | controller | - ' echo ${i}' 2026-01-21 11:38:24.436254 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.436262 | controller | - ' fi' 2026-01-21 11:38:24.436270 | controller | - ' done' 2026-01-21 11:38:24.436278 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.436286 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.436294 | controller | - ' exit 1' 2026-01-21 11:38:24.436302 | controller | - ' else' 2026-01-21 11:38:24.436310 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.436318 | controller | - ' fi' 2026-01-21 11:38:24.436329 | controller | - ' ]' 2026-01-21 11:38:24.436337 | controller | - ' logger.go:42: 11:35:43 | watcher-api-scaling/1-deploy-with-defaults | + oc 2026-01-21 11:38:24.436344 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.436352 | controller | -c Name -c Type' 2026-01-21 11:38:24.436374 | controller | - ' logger.go:42: 11:35:43 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.436384 | controller | -c ''^watcher''' 2026-01-21 11:38:24.436391 | controller | - ' logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436399 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.436407 | controller | - ' logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults | ++ awk 2026-01-21 11:38:24.436414 | controller | ''{print $1}''' 2026-01-21 11:38:24.436422 | controller | - ' logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436429 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.436437 | controller | -c Name -c Type -c ID' 2026-01-21 11:38:24.436445 | controller | - ' logger.go:42: 11:35:46 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.436452 | controller | watcher' 2026-01-21 11:38:24.436460 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c' 2026-01-21 11:38:24.436467 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436474 | controller | get -n watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.436481 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436490 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c '']''' 2026-01-21 11:38:24.436497 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436505 | controller | get -n watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.436513 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436520 | controller | -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.436528 | controller | '']''' 2026-01-21 11:38:24.436535 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436543 | controller | get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.436550 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436558 | controller | '''' == '''' '']''' 2026-01-21 11:38:24.436566 | controller | - ' logger.go:42: 11:35:50 | watcher-api-scaling/1-deploy-with-defaults | + exit 2026-01-21 11:38:24.436587 | controller | 0' 2026-01-21 11:38:24.436596 | controller | - ' logger.go:42: 11:35:52 | watcher-api-scaling/1-deploy-with-defaults | running 2026-01-21 11:38:24.436603 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.436611 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.436618 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.436626 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.436634 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.436641 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.436649 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.436656 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.436664 | controller | ]' 2026-01-21 11:38:24.436672 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.436680 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.436688 | controller | == "" ]; then' 2026-01-21 11:38:24.436694 | controller | - ' exit 0' 2026-01-21 11:38:24.436700 | controller | - ' fi' 2026-01-21 11:38:24.436734 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.436745 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.436751 | controller | - ' counter=0' 2026-01-21 11:38:24.436757 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.436764 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.436770 | controller | - ' echo ${i}' 2026-01-21 11:38:24.436776 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.436781 | controller | - ' fi' 2026-01-21 11:38:24.436787 | controller | - ' done' 2026-01-21 11:38:24.436793 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.436799 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.436805 | controller | - ' exit 1' 2026-01-21 11:38:24.436811 | controller | - ' else' 2026-01-21 11:38:24.436817 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.436823 | controller | - ' fi' 2026-01-21 11:38:24.436829 | controller | - ' ]' 2026-01-21 11:38:24.436835 | controller | - ' logger.go:42: 11:35:52 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.436841 | controller | -c ''^watcher''' 2026-01-21 11:38:24.436847 | controller | - ' logger.go:42: 11:35:52 | watcher-api-scaling/1-deploy-with-defaults | + oc 2026-01-21 11:38:24.436853 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.436859 | controller | -c Name -c Type' 2026-01-21 11:38:24.436865 | controller | - ' logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436870 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.436876 | controller | - ' logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436882 | controller | exec -n watcher-kuttl-default openstackclient -- openstack service list -f value 2026-01-21 11:38:24.436888 | controller | -c Name -c Type -c ID' 2026-01-21 11:38:24.436894 | controller | - ' logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults | ++ awk 2026-01-21 11:38:24.436900 | controller | ''{print $1}''' 2026-01-21 11:38:24.436906 | controller | - ' logger.go:42: 11:35:55 | watcher-api-scaling/1-deploy-with-defaults | ++ grep 2026-01-21 11:38:24.436912 | controller | watcher' 2026-01-21 11:38:24.436918 | controller | - ' logger.go:42: 11:35:58 | watcher-api-scaling/1-deploy-with-defaults | + SERVICEID=ccb89c692c9b4228aec3cccf888acd3c' 2026-01-21 11:38:24.436924 | controller | - ' logger.go:42: 11:35:58 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436934 | controller | get -n watcher-kuttl-default keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.436939 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436945 | controller | ccb89c692c9b4228aec3cccf888acd3c == ccb89c692c9b4228aec3cccf888acd3c '']''' 2026-01-21 11:38:24.436963 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436970 | controller | get -n watcher-kuttl-default watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.436976 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.436982 | controller | -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.436987 | controller | '']''' 2026-01-21 11:38:24.436993 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | ++ oc 2026-01-21 11:38:24.436999 | controller | get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.437005 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | + ''['' 2026-01-21 11:38:24.437011 | controller | '''' == '''' '']''' 2026-01-21 11:38:24.437017 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | + exit 2026-01-21 11:38:24.437023 | controller | 0' 2026-01-21 11:38:24.437029 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/1-deploy-with-defaults | test 2026-01-21 11:38:24.437035 | controller | step completed 1-deploy-with-defaults' 2026-01-21 11:38:24.437041 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/2-scale-up-watcher-api | starting 2026-01-21 11:38:24.437047 | controller | test step 2-scale-up-watcher-api' 2026-01-21 11:38:24.437053 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/2-scale-up-watcher-api | running 2026-01-21 11:38:24.437058 | controller | command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type=''json'' -p=''[{"op": 2026-01-21 11:38:24.437064 | controller | "replace", "path": "/spec/apiServiceTemplate/replicas", "value":3}]''' 2026-01-21 11:38:24.437070 | controller | - ' ]' 2026-01-21 11:38:24.437076 | controller | - ' logger.go:42: 11:35:59 | watcher-api-scaling/2-scale-up-watcher-api | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.437082 | controller | patched' 2026-01-21 11:38:24.437088 | controller | - ' logger.go:42: 11:36:12 | watcher-api-scaling/2-scale-up-watcher-api | test 2026-01-21 11:38:24.437094 | controller | step completed 2-scale-up-watcher-api' 2026-01-21 11:38:24.437100 | controller | - ' logger.go:42: 11:36:12 | watcher-api-scaling/3-scale-down-watcher-api | starting 2026-01-21 11:38:24.437106 | controller | test step 3-scale-down-watcher-api' 2026-01-21 11:38:24.437112 | controller | - ' logger.go:42: 11:36:12 | watcher-api-scaling/3-scale-down-watcher-api | running 2026-01-21 11:38:24.437118 | controller | command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type=''json'' -p=''[{"op": 2026-01-21 11:38:24.437124 | controller | "replace", "path": "/spec/apiServiceTemplate/replicas", "value":1}]''' 2026-01-21 11:38:24.437130 | controller | - ' ]' 2026-01-21 11:38:24.437144 | controller | - ' logger.go:42: 11:36:12 | watcher-api-scaling/3-scale-down-watcher-api | watcher.watcher.openstack.org/watcher-kuttl 2026-01-21 11:38:24.437151 | controller | patched' 2026-01-21 11:38:24.437157 | controller | - ' logger.go:42: 11:36:16 | watcher-api-scaling/3-scale-down-watcher-api | test 2026-01-21 11:38:24.437163 | controller | step completed 3-scale-down-watcher-api' 2026-01-21 11:38:24.437169 | controller | - ' logger.go:42: 11:36:16 | watcher-api-scaling/4-scale-down-zero-watcher-api 2026-01-21 11:38:24.437175 | controller | | starting test step 4-scale-down-zero-watcher-api' 2026-01-21 11:38:24.437182 | controller | - ' logger.go:42: 11:36:16 | watcher-api-scaling/4-scale-down-zero-watcher-api 2026-01-21 11:38:24.437188 | controller | | running command: [sh -c oc patch watcher -n $NAMESPACE watcher-kuttl --type=''json'' 2026-01-21 11:38:24.437205 | controller | -p=''[{"op": "replace", "path": "/spec/apiServiceTemplate/replicas", "value":0}]''' 2026-01-21 11:38:24.437212 | controller | - ' ]' 2026-01-21 11:38:24.437217 | controller | - ' logger.go:42: 11:36:16 | watcher-api-scaling/4-scale-down-zero-watcher-api 2026-01-21 11:38:24.437223 | controller | | watcher.watcher.openstack.org/watcher-kuttl patched' 2026-01-21 11:38:24.437229 | controller | - ' logger.go:42: 11:36:17 | watcher-api-scaling/4-scale-down-zero-watcher-api 2026-01-21 11:38:24.437235 | controller | | test step completed 4-scale-down-zero-watcher-api' 2026-01-21 11:38:24.437241 | controller | - ' logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher | starting test 2026-01-21 11:38:24.437247 | controller | step 5-cleanup-watcher' 2026-01-21 11:38:24.437253 | controller | - ' logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher | running command: 2026-01-21 11:38:24.437259 | controller | [sh -c set -ex' 2026-01-21 11:38:24.437265 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.437270 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.437276 | controller | - ' ]' 2026-01-21 11:38:24.437282 | controller | - ' logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher | + oc exec 2026-01-21 11:38:24.437288 | controller | -n watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.437294 | controller | -c Type' 2026-01-21 11:38:24.437300 | controller | - ' logger.go:42: 11:36:17 | watcher-api-scaling/5-cleanup-watcher | ++ grep -c 2026-01-21 11:38:24.437306 | controller | ''^watcher''' 2026-01-21 11:38:24.437311 | controller | - ' logger.go:42: 11:36:21 | watcher-api-scaling/5-cleanup-watcher | + ''['' 1 2026-01-21 11:38:24.437317 | controller | == 0 '']''' 2026-01-21 11:38:24.437323 | controller | - ' logger.go:42: 11:36:23 | watcher-api-scaling/5-cleanup-watcher | running command: 2026-01-21 11:38:24.437329 | controller | [sh -c set -ex' 2026-01-21 11:38:24.437335 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.437341 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.437347 | controller | - ' ]' 2026-01-21 11:38:24.437353 | controller | - ' logger.go:42: 11:36:23 | watcher-api-scaling/5-cleanup-watcher | + oc exec 2026-01-21 11:38:24.437358 | controller | -n watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.437364 | controller | -c Type' 2026-01-21 11:38:24.437370 | controller | - ' logger.go:42: 11:36:23 | watcher-api-scaling/5-cleanup-watcher | ++ grep -c 2026-01-21 11:38:24.437376 | controller | ''^watcher''' 2026-01-21 11:38:24.437389 | controller | - ' logger.go:42: 11:36:27 | watcher-api-scaling/5-cleanup-watcher | + ''['' 0 2026-01-21 11:38:24.437395 | controller | == 0 '']''' 2026-01-21 11:38:24.437401 | controller | - ' logger.go:42: 11:36:28 | watcher-api-scaling/5-cleanup-watcher | running command: 2026-01-21 11:38:24.437407 | controller | [sh -c set -ex' 2026-01-21 11:38:24.437413 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.437419 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.437425 | controller | - ' ]' 2026-01-21 11:38:24.437431 | controller | - ' logger.go:42: 11:36:28 | watcher-api-scaling/5-cleanup-watcher | + oc exec 2026-01-21 11:38:24.437437 | controller | -n watcher-kuttl-default openstackclient -- openstack service list -f value -c Name 2026-01-21 11:38:24.437442 | controller | -c Type' 2026-01-21 11:38:24.437448 | controller | - ' logger.go:42: 11:36:28 | watcher-api-scaling/5-cleanup-watcher | ++ grep -c 2026-01-21 11:38:24.437454 | controller | ''^watcher''' 2026-01-21 11:38:24.437460 | controller | - ' logger.go:42: 11:36:32 | watcher-api-scaling/5-cleanup-watcher | + ''['' 0 2026-01-21 11:38:24.437466 | controller | == 0 '']''' 2026-01-21 11:38:24.437472 | controller | - ' logger.go:42: 11:36:32 | watcher-api-scaling/5-cleanup-watcher | test step 2026-01-21 11:38:24.437478 | controller | completed 5-cleanup-watcher' 2026-01-21 11:38:24.437484 | controller | - ' logger.go:42: 11:36:32 | watcher-api-scaling | skipping kubernetes event logging' 2026-01-21 11:38:24.437493 | controller | - === CONT kuttl/harness/watcher 2026-01-21 11:38:24.437499 | controller | - ' logger.go:42: 11:36:32 | watcher | Skipping creation of user-supplied namespace: 2026-01-21 11:38:24.437505 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.437511 | controller | - ' logger.go:42: 11:36:32 | watcher/0-cleanup-watcher | starting test step 0-cleanup-watcher' 2026-01-21 11:38:24.437517 | controller | - ' logger.go:42: 11:36:32 | watcher/0-cleanup-watcher | test step completed 0-cleanup-watcher' 2026-01-21 11:38:24.437523 | controller | - ' logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults | starting test step 2026-01-21 11:38:24.437528 | controller | 1-deploy-with-defaults' 2026-01-21 11:38:24.437534 | controller | - ' logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.437540 | controller | created' 2026-01-21 11:38:24.437546 | controller | - ' logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults | running command: 2026-01-21 11:38:24.437552 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.437567 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.437573 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.437579 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.437585 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.437591 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.437597 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.437603 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.437609 | controller | ]' 2026-01-21 11:38:24.437615 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.437649 | controller | -o jsonpath=''{.data.my\.cnf}''|base64 -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.437662 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.437671 | controller | -o jsonpath=''{.data.00-default\.conf}''|base64 -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" 2026-01-21 11:38:24.437679 | controller | == 2 ]' 2026-01-21 11:38:24.437687 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.437694 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.437701 | controller | == "" ]; then' 2026-01-21 11:38:24.437741 | controller | - ' exit 0' 2026-01-21 11:38:24.437752 | controller | - ' fi' 2026-01-21 11:38:24.437759 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.437765 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.437771 | controller | - ' counter=0' 2026-01-21 11:38:24.437777 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.437783 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.437789 | controller | - ' echo ${i}' 2026-01-21 11:38:24.437795 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.437803 | controller | - ' fi' 2026-01-21 11:38:24.437810 | controller | - ' done' 2026-01-21 11:38:24.437818 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.437826 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.437834 | controller | - ' exit 1' 2026-01-21 11:38:24.437841 | controller | - ' else' 2026-01-21 11:38:24.437849 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.437857 | controller | - ' fi' 2026-01-21 11:38:24.437865 | controller | - ' ]' 2026-01-21 11:38:24.437872 | controller | - ' logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.437894 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.437902 | controller | - ' logger.go:42: 11:36:32 | watcher/1-deploy-with-defaults | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.437908 | controller | - ' logger.go:42: 11:36:36 | watcher/1-deploy-with-defaults | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.437914 | controller | - ' logger.go:42: 11:36:38 | watcher/1-deploy-with-defaults | running command: 2026-01-21 11:38:24.437921 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.437929 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.437936 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.437944 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.437952 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.437960 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.437967 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.437975 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.437988 | controller | ]' 2026-01-21 11:38:24.437995 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.438002 | controller | -o jsonpath=''{.data.my\.cnf}''|base64 -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.438009 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.438017 | controller | -o jsonpath=''{.data.00-default\.conf}''|base64 -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" 2026-01-21 11:38:24.438024 | controller | == 2 ]' 2026-01-21 11:38:24.438030 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.438036 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.438042 | controller | == "" ]; then' 2026-01-21 11:38:24.438048 | controller | - ' exit 0' 2026-01-21 11:38:24.438053 | controller | - ' fi' 2026-01-21 11:38:24.438059 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.438065 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.438071 | controller | - ' counter=0' 2026-01-21 11:38:24.438077 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.438083 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.438089 | controller | - ' echo ${i}' 2026-01-21 11:38:24.438095 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.438101 | controller | - ' fi' 2026-01-21 11:38:24.438106 | controller | - ' done' 2026-01-21 11:38:24.438127 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.438134 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.438140 | controller | - ' exit 1' 2026-01-21 11:38:24.438146 | controller | - ' else' 2026-01-21 11:38:24.438152 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.438159 | controller | - ' fi' 2026-01-21 11:38:24.438168 | controller | - ' ]' 2026-01-21 11:38:24.438174 | controller | - ' logger.go:42: 11:36:38 | watcher/1-deploy-with-defaults | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.438180 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.438186 | controller | - ' logger.go:42: 11:36:38 | watcher/1-deploy-with-defaults | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.438192 | controller | - ' logger.go:42: 11:36:41 | watcher/1-deploy-with-defaults | + ''['' 0 == 1 '']''' 2026-01-21 11:38:24.438198 | controller | - ' logger.go:42: 11:36:42 | watcher/1-deploy-with-defaults | running command: 2026-01-21 11:38:24.438216 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.438222 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.438228 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.438243 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.438250 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.438257 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.438263 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.438271 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.438277 | controller | ]' 2026-01-21 11:38:24.438284 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.438291 | controller | -o jsonpath=''{.data.my\.cnf}''|base64 -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.438297 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.438303 | controller | -o jsonpath=''{.data.00-default\.conf}''|base64 -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" 2026-01-21 11:38:24.438309 | controller | == 2 ]' 2026-01-21 11:38:24.438315 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.438321 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.438327 | controller | == "" ]; then' 2026-01-21 11:38:24.438333 | controller | - ' exit 0' 2026-01-21 11:38:24.438339 | controller | - ' fi' 2026-01-21 11:38:24.438345 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.438350 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.438356 | controller | - ' counter=0' 2026-01-21 11:38:24.438362 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.438368 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.438374 | controller | - ' echo ${i}' 2026-01-21 11:38:24.438380 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.438386 | controller | - ' fi' 2026-01-21 11:38:24.438392 | controller | - ' done' 2026-01-21 11:38:24.438398 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.438404 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.438410 | controller | - ' exit 1' 2026-01-21 11:38:24.438415 | controller | - ' else' 2026-01-21 11:38:24.438421 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.438427 | controller | - ' fi' 2026-01-21 11:38:24.438433 | controller | - ' ]' 2026-01-21 11:38:24.438439 | controller | - ' logger.go:42: 11:36:42 | watcher/1-deploy-with-defaults | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.438445 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.438451 | controller | - ' logger.go:42: 11:36:42 | watcher/1-deploy-with-defaults | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.438457 | controller | - ' logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.438463 | controller | - ' logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.438471 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.438478 | controller | - ' logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults | ++ grep watcher' 2026-01-21 11:38:24.438484 | controller | - ' logger.go:42: 11:36:45 | watcher/1-deploy-with-defaults | ++ awk ''{print $1}''' 2026-01-21 11:38:24.438490 | controller | - ' logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df' 2026-01-21 11:38:24.438506 | controller | - ' logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.438512 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.438517 | controller | - ' logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | + ''['' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.438523 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df '']''' 2026-01-21 11:38:24.438529 | controller | - ' logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.438535 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.438553 | controller | - ' logger.go:42: 11:36:48 | watcher/1-deploy-with-defaults | + ''['' -n '''' '']''' 2026-01-21 11:38:24.438559 | controller | - ' logger.go:42: 11:36:49 | watcher/1-deploy-with-defaults | running command: 2026-01-21 11:38:24.438565 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.438571 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.438577 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.438583 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.438602 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.438610 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.438617 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.438623 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.438629 | controller | ]' 2026-01-21 11:38:24.438635 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.438650 | controller | -o jsonpath=''{.data.my\.cnf}''|base64 -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.438656 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.438662 | controller | -o jsonpath=''{.data.00-default\.conf}''|base64 -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" 2026-01-21 11:38:24.438668 | controller | == 2 ]' 2026-01-21 11:38:24.438683 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.438690 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.438696 | controller | == "" ]; then' 2026-01-21 11:38:24.438701 | controller | - ' exit 0' 2026-01-21 11:38:24.438707 | controller | - ' fi' 2026-01-21 11:38:24.438727 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.438737 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.438746 | controller | - ' counter=0' 2026-01-21 11:38:24.438752 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.438758 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.438765 | controller | - ' echo ${i}' 2026-01-21 11:38:24.438771 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.438776 | controller | - ' fi' 2026-01-21 11:38:24.438783 | controller | - ' done' 2026-01-21 11:38:24.438789 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.438795 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.438801 | controller | - ' exit 1' 2026-01-21 11:38:24.438806 | controller | - ' else' 2026-01-21 11:38:24.438812 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.438818 | controller | - ' fi' 2026-01-21 11:38:24.438824 | controller | - ' ]' 2026-01-21 11:38:24.438830 | controller | - ' logger.go:42: 11:36:49 | watcher/1-deploy-with-defaults | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.438840 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.438846 | controller | - ' logger.go:42: 11:36:49 | watcher/1-deploy-with-defaults | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.438852 | controller | - ' logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.438858 | controller | - ' logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.438864 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.438869 | controller | - ' logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults | ++ grep watcher' 2026-01-21 11:38:24.438875 | controller | - ' logger.go:42: 11:36:53 | watcher/1-deploy-with-defaults | ++ awk ''{print $1}''' 2026-01-21 11:38:24.438881 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df' 2026-01-21 11:38:24.438887 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.438893 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.438899 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | + ''['' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.438905 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df '']''' 2026-01-21 11:38:24.438911 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.438917 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.438933 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.438943 | controller | '']''' 2026-01-21 11:38:24.438951 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ grep -c ssl=1' 2026-01-21 11:38:24.438958 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ base64 -d' 2026-01-21 11:38:24.438966 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.438973 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.my\.cnf}''' 2026-01-21 11:38:24.438981 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.439005 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ grep -c ''cafile 2026-01-21 11:38:24.439014 | controller | = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem''' 2026-01-21 11:38:24.439021 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ base64 -d' 2026-01-21 11:38:24.439028 | controller | - ' logger.go:42: 11:36:57 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439035 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.00-default\.conf}''' 2026-01-21 11:38:24.439043 | controller | - ' logger.go:42: 11:36:58 | watcher/1-deploy-with-defaults | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.439050 | controller | - ' logger.go:42: 11:36:58 | watcher/1-deploy-with-defaults | ++ oc get pods -n 2026-01-21 11:38:24.439058 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.439070 | controller | - ' logger.go:42: 11:36:58 | watcher/1-deploy-with-defaults | + ''['' '''' == '''' 2026-01-21 11:38:24.439077 | controller | '']''' 2026-01-21 11:38:24.439085 | controller | - ' logger.go:42: 11:36:58 | watcher/1-deploy-with-defaults | + exit 0' 2026-01-21 11:38:24.439092 | controller | - ' logger.go:42: 11:36:59 | watcher/1-deploy-with-defaults | running command: 2026-01-21 11:38:24.439100 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.439107 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.439115 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.439123 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.439147 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.439154 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.439162 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.439175 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.439185 | controller | ]' 2026-01-21 11:38:24.439193 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.439202 | controller | -o jsonpath=''{.data.my\.cnf}''|base64 -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.439210 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.439218 | controller | -o jsonpath=''{.data.00-default\.conf}''|base64 -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" 2026-01-21 11:38:24.439225 | controller | == 2 ]' 2026-01-21 11:38:24.439233 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.439240 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.439248 | controller | == "" ]; then' 2026-01-21 11:38:24.439255 | controller | - ' exit 0' 2026-01-21 11:38:24.439262 | controller | - ' fi' 2026-01-21 11:38:24.439269 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.439276 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.439284 | controller | - ' counter=0' 2026-01-21 11:38:24.439291 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.439298 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.439305 | controller | - ' echo ${i}' 2026-01-21 11:38:24.439312 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.439320 | controller | - ' fi' 2026-01-21 11:38:24.439328 | controller | - ' done' 2026-01-21 11:38:24.439335 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.439343 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.439350 | controller | - ' exit 1' 2026-01-21 11:38:24.439357 | controller | - ' else' 2026-01-21 11:38:24.439364 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.439371 | controller | - ' fi' 2026-01-21 11:38:24.439377 | controller | - ' ]' 2026-01-21 11:38:24.439382 | controller | - ' logger.go:42: 11:36:59 | watcher/1-deploy-with-defaults | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.439388 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.439394 | controller | - ' logger.go:42: 11:36:59 | watcher/1-deploy-with-defaults | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.439400 | controller | - ' logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.439411 | controller | - ' logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.439417 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.439423 | controller | - ' logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults | ++ awk ''{print $1}''' 2026-01-21 11:38:24.439429 | controller | - ' logger.go:42: 11:37:02 | watcher/1-deploy-with-defaults | ++ grep watcher' 2026-01-21 11:38:24.439435 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df' 2026-01-21 11:38:24.439440 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439446 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.439452 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + ''['' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.439475 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df '']''' 2026-01-21 11:38:24.439494 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439500 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.439506 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.439512 | controller | '']''' 2026-01-21 11:38:24.439518 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439524 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.my\.cnf}''' 2026-01-21 11:38:24.439530 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ base64 -d' 2026-01-21 11:38:24.439536 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ grep -c ssl=1' 2026-01-21 11:38:24.439542 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.439548 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ base64 -d' 2026-01-21 11:38:24.439554 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ grep -c ''cafile 2026-01-21 11:38:24.439560 | controller | = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem''' 2026-01-21 11:38:24.439565 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439571 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.00-default\.conf}''' 2026-01-21 11:38:24.439577 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.439583 | controller | - ' logger.go:42: 11:37:05 | watcher/1-deploy-with-defaults | ++ oc get pods -n 2026-01-21 11:38:24.439589 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.439595 | controller | - ' logger.go:42: 11:37:06 | watcher/1-deploy-with-defaults | + ''['' '''' == '''' 2026-01-21 11:38:24.439600 | controller | '']''' 2026-01-21 11:38:24.439606 | controller | - ' logger.go:42: 11:37:06 | watcher/1-deploy-with-defaults | + exit 0' 2026-01-21 11:38:24.439612 | controller | - ' logger.go:42: 11:37:07 | watcher/1-deploy-with-defaults | running command: 2026-01-21 11:38:24.439618 | controller | [sh -c set -euxo pipefail' 2026-01-21 11:38:24.439624 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.439630 | controller | -f value -c Name -c Type |[ $(grep -c ^watcher) == 1 ]' 2026-01-21 11:38:24.439636 | controller | - ' SERVICEID=$(oc exec -n watcher-kuttl-default openstackclient -- openstack 2026-01-21 11:38:24.439641 | controller | service list -f value -c Name -c Type -c ID | grep watcher| awk ''{print $1}'')' 2026-01-21 11:38:24.439647 | controller | - ' [ $(oc get -n watcher-kuttl-default keystoneservice watcher -o jsonpath={.status.serviceID}) 2026-01-21 11:38:24.439653 | controller | == ${SERVICEID} ]' 2026-01-21 11:38:24.439659 | controller | - ' [ -n "$(oc get -n watcher-kuttl-default watcher watcher-kuttl -o jsonpath={.status.hash.dbsync})" 2026-01-21 11:38:24.439665 | controller | ]' 2026-01-21 11:38:24.439671 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.439676 | controller | -o jsonpath=''{.data.my\.cnf}''|base64 -d|grep -c ''ssl=1'')" == 1 ]' 2026-01-21 11:38:24.439682 | controller | - ' [ "$(oc get -n watcher-kuttl-default secret watcher-kuttl-api-config-data 2026-01-21 11:38:24.439688 | controller | -o jsonpath=''{.data.00-default\.conf}''|base64 -d|grep -c ''cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem'')" 2026-01-21 11:38:24.439694 | controller | == 2 ]' 2026-01-21 11:38:24.439700 | controller | - ' # If we are running the container locally, skip following test' 2026-01-21 11:38:24.439706 | controller | - ' if [ "$(oc get pods -n openstack-operators -o name -l openstack.org/operator-name=watcher)" 2026-01-21 11:38:24.439731 | controller | == "" ]; then' 2026-01-21 11:38:24.439741 | controller | - ' exit 0' 2026-01-21 11:38:24.439747 | controller | - ' fi' 2026-01-21 11:38:24.439753 | controller | - ' env_variables=$(oc set env $(oc get pods -n openstack-operators -o name 2026-01-21 11:38:24.439758 | controller | -l openstack.org/operator-name=watcher) -n openstack-operators --list)' 2026-01-21 11:38:24.439764 | controller | - ' counter=0' 2026-01-21 11:38:24.439770 | controller | - ' for i in ${env_variables}; do' 2026-01-21 11:38:24.439776 | controller | - ' if echo ${i} | grep ''_URL_DEFAULT'' &> /dev/null; then' 2026-01-21 11:38:24.439782 | controller | - ' echo ${i}' 2026-01-21 11:38:24.439788 | controller | - ' counter=$((counter + 1))' 2026-01-21 11:38:24.439794 | controller | - ' fi' 2026-01-21 11:38:24.439800 | controller | - ' done' 2026-01-21 11:38:24.439806 | controller | - ' if [ ${counter} -lt 3 ]; then' 2026-01-21 11:38:24.439812 | controller | - ' echo "Error: Less than 3 _URL_DEFAULT variables found."' 2026-01-21 11:38:24.439818 | controller | - ' exit 1' 2026-01-21 11:38:24.439823 | controller | - ' else' 2026-01-21 11:38:24.439829 | controller | - ' echo "Success: ${counter} _URL_DEFAULT variables found."' 2026-01-21 11:38:24.439835 | controller | - ' fi' 2026-01-21 11:38:24.439841 | controller | - ' ]' 2026-01-21 11:38:24.439847 | controller | - ' logger.go:42: 11:37:07 | watcher/1-deploy-with-defaults | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.439853 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.439859 | controller | - ' logger.go:42: 11:37:07 | watcher/1-deploy-with-defaults | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.439880 | controller | - ' logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.439887 | controller | - ' logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults | ++ grep watcher' 2026-01-21 11:38:24.439892 | controller | - ' logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults | ++ awk ''{print $1}''' 2026-01-21 11:38:24.439898 | controller | - ' logger.go:42: 11:37:09 | watcher/1-deploy-with-defaults | ++ oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.439904 | controller | openstackclient -- openstack service list -f value -c Name -c Type -c ID' 2026-01-21 11:38:24.439910 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | + SERVICEID=33c8cb4c3b7c424a9077551e0a73e2df' 2026-01-21 11:38:24.439916 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439922 | controller | keystoneservice watcher -o ''jsonpath={.status.serviceID}''' 2026-01-21 11:38:24.439928 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | + ''['' 33c8cb4c3b7c424a9077551e0a73e2df 2026-01-21 11:38:24.439934 | controller | == 33c8cb4c3b7c424a9077551e0a73e2df '']''' 2026-01-21 11:38:24.439939 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439945 | controller | watcher watcher-kuttl -o ''jsonpath={.status.hash.dbsync}''' 2026-01-21 11:38:24.439951 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | + ''['' -n n99h64chd6h589h79h659h5d7h5b5hffh64dh5fch7dh59ch9bh77h54bhb9h655h576h5b6h9h57dh5h558h577h5cdhbfh5fbh659hd4h5fdhdcq 2026-01-21 11:38:24.439957 | controller | '']''' 2026-01-21 11:38:24.439963 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.439969 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.my\.cnf}''' 2026-01-21 11:38:24.439975 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ base64 -d' 2026-01-21 11:38:24.439981 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ grep -c ssl=1' 2026-01-21 11:38:24.439987 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | + ''['' 1 == 1 '']''' 2026-01-21 11:38:24.439993 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ base64 -d' 2026-01-21 11:38:24.440002 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ grep -c ''cafile 2026-01-21 11:38:24.440008 | controller | = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem''' 2026-01-21 11:38:24.440014 | controller | - ' logger.go:42: 11:37:12 | watcher/1-deploy-with-defaults | ++ oc get -n watcher-kuttl-default 2026-01-21 11:38:24.440020 | controller | secret watcher-kuttl-api-config-data -o ''jsonpath={.data.00-default\.conf}''' 2026-01-21 11:38:24.440026 | controller | - ' logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults | + ''['' 2 == 2 '']''' 2026-01-21 11:38:24.440031 | controller | - ' logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults | ++ oc get pods -n 2026-01-21 11:38:24.440037 | controller | openstack-operators -o name -l openstack.org/operator-name=watcher' 2026-01-21 11:38:24.440043 | controller | - ' logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults | + ''['' '''' == '''' 2026-01-21 11:38:24.440049 | controller | '']''' 2026-01-21 11:38:24.440055 | controller | - ' logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults | + exit 0' 2026-01-21 11:38:24.440061 | controller | - ' logger.go:42: 11:37:13 | watcher/1-deploy-with-defaults | test step completed 2026-01-21 11:38:24.440066 | controller | 1-deploy-with-defaults' 2026-01-21 11:38:24.440072 | controller | - ' logger.go:42: 11:37:13 | watcher/2-cleanup-watcher | starting test step 2-cleanup-watcher' 2026-01-21 11:38:24.440078 | controller | - ' logger.go:42: 11:37:13 | watcher/2-cleanup-watcher | test step completed 2-cleanup-watcher' 2026-01-21 11:38:24.440084 | controller | - ' logger.go:42: 11:37:13 | watcher/3-precreate-mariadbaccount | starting test 2026-01-21 11:38:24.440090 | controller | step 3-precreate-mariadbaccount' 2026-01-21 11:38:24.440099 | controller | - ' logger.go:42: 11:37:13 | watcher/3-precreate-mariadbaccount | MariaDBAccount:watcher-kuttl-default/watcher-precreated 2026-01-21 11:38:24.440105 | controller | created' 2026-01-21 11:38:24.440112 | controller | - ' logger.go:42: 11:37:13 | watcher/3-precreate-mariadbaccount | test step completed 2026-01-21 11:38:24.440118 | controller | 3-precreate-mariadbaccount' 2026-01-21 11:38:24.440124 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | starting 2026-01-21 11:38:24.440129 | controller | test step 4-deploy-with-precreated-account' 2026-01-21 11:38:24.440135 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | Secret:watcher-kuttl-default/custom-prometheus-config 2026-01-21 11:38:24.440142 | controller | created' 2026-01-21 11:38:24.440148 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | Watcher:watcher-kuttl-default/watcher-kuttl 2026-01-21 11:38:24.440153 | controller | created' 2026-01-21 11:38:24.440159 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.440165 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.440171 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.440177 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.440183 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.440189 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.440203 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.440209 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.440215 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.440221 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.440227 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.440235 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.440246 | controller | == 1 ]' 2026-01-21 11:38:24.440252 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.440258 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.440276 | controller | - ' else' 2026-01-21 11:38:24.440283 | controller | - ' exit 1' 2026-01-21 11:38:24.440291 | controller | - ' fi' 2026-01-21 11:38:24.440298 | controller | - ' ]' 2026-01-21 11:38:24.440306 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.440314 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.440322 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.440329 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.440335 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.440341 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.440346 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.440352 | controller | -1' 2026-01-21 11:38:24.440358 | controller | - ' logger.go:42: 11:37:13 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.440364 | controller | -v ''^$''' 2026-01-21 11:38:24.440370 | controller | - ' logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.440376 | controller | - ' logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.440382 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.440388 | controller | - ' logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.440394 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.440399 | controller | - ' logger.go:42: 11:37:14 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.440405 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.440411 | controller | - ' logger.go:42: 11:37:15 | watcher/4-deploy-with-precreated-account | error: 2026-01-21 11:38:24.440417 | controller | Internal error occurred: error executing command in container: container is not 2026-01-21 11:38:24.440423 | controller | created or running' 2026-01-21 11:38:24.440429 | controller | - ' logger.go:42: 11:37:15 | watcher/4-deploy-with-precreated-account | ++ echo' 2026-01-21 11:38:24.440435 | controller | - ' logger.go:42: 11:37:15 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.440441 | controller | 0 == 1 '']''' 2026-01-21 11:38:24.440446 | controller | - ' logger.go:42: 11:37:16 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.440452 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.440462 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.440468 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.440474 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.440479 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.440485 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.440491 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.440497 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.440503 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.440509 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.440515 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.440521 | controller | == 1 ]' 2026-01-21 11:38:24.440530 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.440536 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.440543 | controller | - ' else' 2026-01-21 11:38:24.440549 | controller | - ' exit 1' 2026-01-21 11:38:24.440554 | controller | - ' fi' 2026-01-21 11:38:24.440560 | controller | - ' ]' 2026-01-21 11:38:24.440566 | controller | - ' logger.go:42: 11:37:16 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.440572 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.440578 | controller | - ' logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.440584 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.440590 | controller | - ' logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.440606 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.440613 | controller | - ' logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.440618 | controller | -v ''^$''' 2026-01-21 11:38:24.440625 | controller | - ' logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.440631 | controller | -1' 2026-01-21 11:38:24.440637 | controller | - ' logger.go:42: 11:37:17 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.440643 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.440649 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.440655 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.440661 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.440667 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.440673 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.440682 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.440688 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.440694 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.440700 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.440706 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.440725 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.440735 | controller | == 1 ]' 2026-01-21 11:38:24.440741 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.440747 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.440753 | controller | - ' else' 2026-01-21 11:38:24.440759 | controller | - ' exit 1' 2026-01-21 11:38:24.440765 | controller | - ' fi' 2026-01-21 11:38:24.440771 | controller | - ' ]' 2026-01-21 11:38:24.440777 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.440782 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.440788 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.440794 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.440800 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.440806 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.440812 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.440818 | controller | -v ''^$''' 2026-01-21 11:38:24.440836 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.440842 | controller | -1' 2026-01-21 11:38:24.440848 | controller | - ' logger.go:42: 11:37:18 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.440854 | controller | - ' logger.go:42: 11:37:19 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.440860 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.440869 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.440875 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.440880 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.440886 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.440892 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.440898 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.440904 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.440910 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.440916 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.440922 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.440928 | controller | == 1 ]' 2026-01-21 11:38:24.440934 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.440940 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.440946 | controller | - ' else' 2026-01-21 11:38:24.440952 | controller | - ' exit 1' 2026-01-21 11:38:24.440958 | controller | - ' fi' 2026-01-21 11:38:24.440964 | controller | - ' ]' 2026-01-21 11:38:24.440970 | controller | - ' logger.go:42: 11:37:19 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.440976 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.440982 | controller | - ' logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.440988 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.440994 | controller | - ' logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.441000 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.441025 | controller | - ' logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.441046 | controller | -1' 2026-01-21 11:38:24.441057 | controller | - ' logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.441066 | controller | -v ''^$''' 2026-01-21 11:38:24.441076 | controller | - ' logger.go:42: 11:37:20 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.441084 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.441093 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.441151 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.441163 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.441171 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.441179 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.441187 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.441195 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.441202 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.441208 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.441218 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.441224 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.441230 | controller | == 1 ]' 2026-01-21 11:38:24.441236 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.441242 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.441248 | controller | - ' else' 2026-01-21 11:38:24.441254 | controller | - ' exit 1' 2026-01-21 11:38:24.441260 | controller | - ' fi' 2026-01-21 11:38:24.441266 | controller | - ' ]' 2026-01-21 11:38:24.441271 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.441277 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.441283 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.441289 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.441295 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.441301 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.441307 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.441313 | controller | -1' 2026-01-21 11:38:24.441321 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.441329 | controller | -v ''^$''' 2026-01-21 11:38:24.441337 | controller | - ' logger.go:42: 11:37:21 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.441345 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.441353 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.441361 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.441368 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.441375 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.441382 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.441388 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.441394 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.441400 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.441406 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.441412 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.441418 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.441424 | controller | == 1 ]' 2026-01-21 11:38:24.441430 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.441436 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.441442 | controller | - ' else' 2026-01-21 11:38:24.441448 | controller | - ' exit 1' 2026-01-21 11:38:24.441454 | controller | - ' fi' 2026-01-21 11:38:24.441460 | controller | - ' ]' 2026-01-21 11:38:24.441466 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.441472 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.441478 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.441484 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.441494 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.441500 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.441506 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.441512 | controller | -1' 2026-01-21 11:38:24.441530 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.441537 | controller | -v ''^$''' 2026-01-21 11:38:24.441543 | controller | - ' logger.go:42: 11:37:23 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.441549 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.441555 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.441561 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.441567 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.441573 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.441579 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.441584 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.441590 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.441597 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.441603 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.441609 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.441615 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.441621 | controller | == 1 ]' 2026-01-21 11:38:24.441627 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.441632 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.441638 | controller | - ' else' 2026-01-21 11:38:24.441644 | controller | - ' exit 1' 2026-01-21 11:38:24.441650 | controller | - ' fi' 2026-01-21 11:38:24.441656 | controller | - ' ]' 2026-01-21 11:38:24.441662 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.441668 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.441674 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.441679 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.441685 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.441691 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.441697 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.441703 | controller | -1' 2026-01-21 11:38:24.441709 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.441736 | controller | -v ''^$''' 2026-01-21 11:38:24.441743 | controller | - ' logger.go:42: 11:37:24 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.441749 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.441755 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.441761 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.441767 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.441773 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.441786 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.441805 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.441812 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.441818 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.441824 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.441832 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.441838 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.441844 | controller | == 1 ]' 2026-01-21 11:38:24.441850 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.441856 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.441862 | controller | - ' else' 2026-01-21 11:38:24.441868 | controller | - ' exit 1' 2026-01-21 11:38:24.441873 | controller | - ' fi' 2026-01-21 11:38:24.441879 | controller | - ' ]' 2026-01-21 11:38:24.441885 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.441891 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.441897 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.441903 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.441909 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.441915 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.441920 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.441926 | controller | -1' 2026-01-21 11:38:24.441941 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.441948 | controller | -v ''^$''' 2026-01-21 11:38:24.441954 | controller | - ' logger.go:42: 11:37:26 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.441960 | controller | - ' logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.441966 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.441972 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.441978 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.441983 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.441989 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.441995 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.442001 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.442007 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.442013 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.442019 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.442025 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.442031 | controller | == 1 ]' 2026-01-21 11:38:24.442038 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.442045 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.442052 | controller | - ' else' 2026-01-21 11:38:24.442059 | controller | - ' exit 1' 2026-01-21 11:38:24.442078 | controller | - ' fi' 2026-01-21 11:38:24.442086 | controller | - ' ]' 2026-01-21 11:38:24.442093 | controller | - ' logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.442101 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.442108 | controller | - ' logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.442116 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.442123 | controller | - ' logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.442130 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.442137 | controller | - ' logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.442144 | controller | -1' 2026-01-21 11:38:24.442150 | controller | - ' logger.go:42: 11:37:27 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.442156 | controller | -v ''^$''' 2026-01-21 11:38:24.442162 | controller | - ' logger.go:42: 11:37:28 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.442168 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.442175 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.442182 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.442190 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.442198 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.442206 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.442214 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.442222 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.442231 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.442242 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.442250 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.442259 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.442269 | controller | == 1 ]' 2026-01-21 11:38:24.442278 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.442289 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.442298 | controller | - ' else' 2026-01-21 11:38:24.442306 | controller | - ' exit 1' 2026-01-21 11:38:24.442314 | controller | - ' fi' 2026-01-21 11:38:24.442321 | controller | - ' ]' 2026-01-21 11:38:24.442329 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.442336 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.442343 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.442350 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.442358 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.442366 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.442375 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.442383 | controller | -1' 2026-01-21 11:38:24.442390 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.442397 | controller | -v ''^$''' 2026-01-21 11:38:24.442421 | controller | - ' logger.go:42: 11:37:29 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.442434 | controller | - ' logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.442440 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.442446 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.442452 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.442458 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.442464 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.442470 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.442476 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.442482 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.442488 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.442494 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.442500 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.442506 | controller | == 1 ]' 2026-01-21 11:38:24.442512 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.442518 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.442524 | controller | - ' else' 2026-01-21 11:38:24.442530 | controller | - ' exit 1' 2026-01-21 11:38:24.442536 | controller | - ' fi' 2026-01-21 11:38:24.442542 | controller | - ' ]' 2026-01-21 11:38:24.442548 | controller | - ' logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.442554 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.442560 | controller | - ' logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.442566 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.442572 | controller | - ' logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.442577 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.442583 | controller | - ' logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.442610 | controller | -1' 2026-01-21 11:38:24.442619 | controller | - ' logger.go:42: 11:37:30 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.442637 | controller | -v ''^$''' 2026-01-21 11:38:24.442646 | controller | - ' logger.go:42: 11:37:31 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.442654 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.442662 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.442689 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.442697 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.442704 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.442789 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.442851 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.442861 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.442868 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.442876 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.442883 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.442891 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.442905 | controller | == 1 ]' 2026-01-21 11:38:24.442912 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.442920 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.442958 | controller | - ' else' 2026-01-21 11:38:24.442966 | controller | - ' exit 1' 2026-01-21 11:38:24.442973 | controller | - ' fi' 2026-01-21 11:38:24.442980 | controller | - ' ]' 2026-01-21 11:38:24.442987 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.442995 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.443002 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.443009 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.443016 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.443024 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.443032 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.443040 | controller | -1' 2026-01-21 11:38:24.443046 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.443054 | controller | -v ''^$''' 2026-01-21 11:38:24.443062 | controller | - ' logger.go:42: 11:37:32 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.443086 | controller | - ' logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.443095 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.443103 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.443110 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.443118 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.443125 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.443138 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.443145 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.443152 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.443160 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.443169 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.443176 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.443183 | controller | == 1 ]' 2026-01-21 11:38:24.443190 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.443198 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.443205 | controller | - ' else' 2026-01-21 11:38:24.443213 | controller | - ' exit 1' 2026-01-21 11:38:24.443221 | controller | - ' fi' 2026-01-21 11:38:24.443228 | controller | - ' ]' 2026-01-21 11:38:24.443236 | controller | - ' logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.443243 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.443251 | controller | - ' logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.443258 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.443266 | controller | - ' logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.443274 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.443287 | controller | - ' logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.443295 | controller | -1' 2026-01-21 11:38:24.443303 | controller | - ' logger.go:42: 11:37:33 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.443311 | controller | -v ''^$''' 2026-01-21 11:38:24.443318 | controller | - ' logger.go:42: 11:37:34 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.443326 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.443333 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.443345 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.443352 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.443363 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.443378 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.443387 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.443395 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.443403 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.443410 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.443418 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.443427 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.443435 | controller | == 1 ]' 2026-01-21 11:38:24.443442 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.443450 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.443465 | controller | - ' else' 2026-01-21 11:38:24.443473 | controller | - ' exit 1' 2026-01-21 11:38:24.443481 | controller | - ' fi' 2026-01-21 11:38:24.443489 | controller | - ' ]' 2026-01-21 11:38:24.443498 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.443507 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.443516 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.443524 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.443532 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.443540 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.443555 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.443567 | controller | -1' 2026-01-21 11:38:24.443574 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.443582 | controller | -v ''^$''' 2026-01-21 11:38:24.443590 | controller | - ' logger.go:42: 11:37:35 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.443622 | controller | - ' logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.443634 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.443642 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.443649 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.443658 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.443666 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.443674 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.443683 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.443697 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.443705 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.443737 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.443752 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.443761 | controller | == 1 ]' 2026-01-21 11:38:24.443769 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.443777 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.443786 | controller | - ' else' 2026-01-21 11:38:24.443794 | controller | - ' exit 1' 2026-01-21 11:38:24.443802 | controller | - ' fi' 2026-01-21 11:38:24.443810 | controller | - ' ]' 2026-01-21 11:38:24.443817 | controller | - ' logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.443824 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.443831 | controller | - ' logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.443838 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.443846 | controller | - ' logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.443855 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.443862 | controller | - ' logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.443870 | controller | -1' 2026-01-21 11:38:24.443877 | controller | - ' logger.go:42: 11:37:36 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.443885 | controller | -v ''^$''' 2026-01-21 11:38:24.443892 | controller | - ' logger.go:42: 11:37:37 | watcher/4-deploy-with-precreated-account | + APIPOD=' 2026-01-21 11:38:24.443900 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.443907 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.443914 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.443923 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.443930 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.443938 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.443946 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.443953 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.443961 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.443969 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.443977 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.443985 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.443992 | controller | == 1 ]' 2026-01-21 11:38:24.444000 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.444007 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.444014 | controller | - ' else' 2026-01-21 11:38:24.444021 | controller | - ' exit 1' 2026-01-21 11:38:24.444029 | controller | - ' fi' 2026-01-21 11:38:24.444036 | controller | - ' ]' 2026-01-21 11:38:24.444042 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.444047 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.444059 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.444065 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.444071 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.444077 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.444082 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.444088 | controller | -v ''^$''' 2026-01-21 11:38:24.444095 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.444102 | controller | -1' 2026-01-21 11:38:24.444118 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.444131 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.444139 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.444163 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.444172 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.444180 | controller | - ' logger.go:42: 11:37:38 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.444189 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.444198 | controller | - ' logger.go:42: 11:37:39 | watcher/4-deploy-with-precreated-account | error: 2026-01-21 11:38:24.444205 | controller | unable to upgrade connection: container not found ("watcher-api")' 2026-01-21 11:38:24.444224 | controller | - ' logger.go:42: 11:37:39 | watcher/4-deploy-with-precreated-account | ++ echo' 2026-01-21 11:38:24.444234 | controller | - ' logger.go:42: 11:37:39 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.444242 | controller | 0 == 1 '']''' 2026-01-21 11:38:24.444250 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.444258 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.444265 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.444273 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.444281 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.444288 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.444295 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.444303 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.444311 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.444318 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.444326 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.444333 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.444341 | controller | == 1 ]' 2026-01-21 11:38:24.444349 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.444356 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.444364 | controller | - ' else' 2026-01-21 11:38:24.444376 | controller | - ' exit 1' 2026-01-21 11:38:24.444384 | controller | - ' fi' 2026-01-21 11:38:24.444391 | controller | - ' ]' 2026-01-21 11:38:24.444399 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.444406 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.444414 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.444426 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.444434 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.444442 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.444450 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.444457 | controller | -1' 2026-01-21 11:38:24.444464 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.444472 | controller | -v ''^$''' 2026-01-21 11:38:24.444479 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.444487 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.444494 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.444502 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.444509 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.444517 | controller | - ' logger.go:42: 11:37:40 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.444525 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.444532 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.444539 | controller | ''#'' Global config' 2026-01-21 11:38:24.444547 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.444554 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.444562 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.444569 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.444577 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.444596 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.444604 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.444612 | controller | ''#'' Service config' 2026-01-21 11:38:24.444620 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.444628 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.444636 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.444662 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.444672 | controller | - ' logger.go:42: 11:37:41 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.444681 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.444690 | controller | - ' logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.444698 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.444707 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.444749 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.444762 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.444770 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.444778 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.444792 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.444800 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.444808 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.444815 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.444823 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.444831 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.444838 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.444846 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.444853 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.444861 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.444923 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.444932 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.444940 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.444947 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.444954 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.444961 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.444968 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.444976 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.444983 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.444991 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.444998 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.445005 | controller | period = 900' 2026-01-21 11:38:24.445019 | controller | - ' logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.445026 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.445033 | controller | - ' logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.445041 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.445048 | controller | - ' logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.445056 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.445063 | controller | - ' logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.445070 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.445084 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.445109 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.445124 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.445132 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.445139 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.445147 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.445154 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.445161 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.445168 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.445175 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.445190 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.445198 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.445205 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.445212 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.445220 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.445228 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.445236 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.445245 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.445252 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.445260 | controller | - ' logger.go:42: 11:37:42 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.445269 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.445284 | controller | - ' logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.445294 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.445301 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.445309 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.445316 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.445324 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.445331 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.445339 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.445347 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.445354 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.445362 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.445369 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.445377 | controller | == 1 ]' 2026-01-21 11:38:24.445385 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.445392 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.445400 | controller | - ' else' 2026-01-21 11:38:24.445407 | controller | - ' exit 1' 2026-01-21 11:38:24.445415 | controller | - ' fi' 2026-01-21 11:38:24.445427 | controller | - ' ]' 2026-01-21 11:38:24.445435 | controller | - ' logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.445443 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.445450 | controller | - ' logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.445458 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.445466 | controller | - ' logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.445473 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.445480 | controller | - ' logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.445487 | controller | -1' 2026-01-21 11:38:24.445508 | controller | - ' logger.go:42: 11:37:43 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.445515 | controller | -v ''^$''' 2026-01-21 11:38:24.445521 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.445527 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.445532 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.445538 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.445555 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.445561 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.445567 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.445573 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.445579 | controller | ''#'' Global config' 2026-01-21 11:38:24.445586 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.445593 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.445600 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.445608 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.445616 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.445623 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.445629 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.445634 | controller | ''#'' Service config' 2026-01-21 11:38:24.445640 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.445646 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.445652 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.445658 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.445664 | controller | - ' logger.go:42: 11:37:44 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.445670 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.445676 | controller | - ' logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.445682 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.445688 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.445694 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.445700 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.445727 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.445739 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.445747 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.445755 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.445763 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.445770 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.445778 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.445786 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.445793 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.445806 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.445820 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.445828 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.445836 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.445844 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.445853 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.445861 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.445869 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.445877 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.445885 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.445908 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.445918 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.445927 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.445936 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.445944 | controller | period = 900' 2026-01-21 11:38:24.445952 | controller | - ' logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.445965 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.445979 | controller | - ' logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.445987 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.445995 | controller | - ' logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.446002 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.446010 | controller | - ' logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.446016 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.446034 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.446039 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.446062 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.446073 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.446081 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.446088 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.446096 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.446103 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.446110 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.446116 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.446128 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.446133 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.446140 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.446148 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.446156 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.446164 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.446172 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.446181 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.446190 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.446197 | controller | - ' logger.go:42: 11:37:45 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.446204 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.446212 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.446219 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.446227 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.446235 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.446242 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.446250 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.446257 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.446265 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.446272 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.446280 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.446287 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.446295 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.446303 | controller | == 1 ]' 2026-01-21 11:38:24.446337 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.446380 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.446387 | controller | - ' else' 2026-01-21 11:38:24.446393 | controller | - ' exit 1' 2026-01-21 11:38:24.446399 | controller | - ' fi' 2026-01-21 11:38:24.446405 | controller | - ' ]' 2026-01-21 11:38:24.446411 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.446416 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.446422 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.446428 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.446434 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.446440 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.446446 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.446452 | controller | -1' 2026-01-21 11:38:24.446458 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.446463 | controller | -v ''^$''' 2026-01-21 11:38:24.446469 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.446475 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.446481 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.446487 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.446493 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.446499 | controller | - ' logger.go:42: 11:37:46 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.446504 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.446510 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.446516 | controller | ''#'' Global config' 2026-01-21 11:38:24.446522 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.446528 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.446534 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.446539 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.446545 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.446551 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.446557 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.446563 | controller | ''#'' Service config' 2026-01-21 11:38:24.446573 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.446579 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.446585 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.446614 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.446622 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.446628 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.446634 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.446640 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.446666 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.446675 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.446683 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.446691 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.446698 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.446704 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.446728 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.446739 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.446747 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.446755 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.446763 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.446785 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.446792 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.446798 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.446804 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.446810 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.446816 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.446822 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.446828 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.446836 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.446844 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.446851 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.446859 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.446867 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.446874 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.446882 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.446888 | controller | period = 900' 2026-01-21 11:38:24.446899 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.446905 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.446911 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.446917 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.446925 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.446933 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.446945 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.446953 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.446968 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.446976 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.446983 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.446991 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.446997 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.447003 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.447010 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.447018 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.447026 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.447033 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.447048 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.447056 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.447063 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.447096 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.447105 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.447112 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.447120 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.447128 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.447135 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.447155 | controller | - ' logger.go:42: 11:37:47 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.447163 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.447171 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.447179 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.447187 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.447194 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.447202 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.447210 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.447217 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.447225 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.447233 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.447242 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.447250 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.447276 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.447287 | controller | == 1 ]' 2026-01-21 11:38:24.447295 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.447303 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.447310 | controller | - ' else' 2026-01-21 11:38:24.447318 | controller | - ' exit 1' 2026-01-21 11:38:24.447325 | controller | - ' fi' 2026-01-21 11:38:24.447333 | controller | - ' ]' 2026-01-21 11:38:24.447341 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.447354 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.447361 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.447369 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.447377 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.447384 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.447391 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.447399 | controller | -1' 2026-01-21 11:38:24.447407 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.447414 | controller | -v ''^$''' 2026-01-21 11:38:24.447422 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.447430 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.447438 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.447445 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.447453 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.447461 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.447468 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.447476 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.447483 | controller | ''#'' Global config' 2026-01-21 11:38:24.447491 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.447499 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.447506 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.447514 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.447521 | controller | - ' logger.go:42: 11:37:49 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.447527 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.447533 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.447539 | controller | ''#'' Service config' 2026-01-21 11:38:24.447545 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.447551 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.447557 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.447563 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.447569 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.447575 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.447594 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.447600 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.447619 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.447626 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.447632 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.447639 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.447644 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.447651 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.447657 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.447662 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.447668 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.447674 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.447680 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.447686 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.447692 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.447698 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.447704 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.447740 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.447749 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.447769 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.447778 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.447785 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.447792 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.447798 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.447804 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.447810 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.447816 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.447822 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.447828 | controller | period = 900' 2026-01-21 11:38:24.447833 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.447839 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.447856 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.447862 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.447868 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.447874 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.447880 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.447886 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.447897 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.447903 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.447909 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.447915 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.447921 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.447927 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.447933 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.447949 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.447955 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.447961 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.447973 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.447979 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.447985 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.447991 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.447997 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.448003 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.448009 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.448015 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.448021 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.448026 | controller | - ' logger.go:42: 11:37:50 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448032 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.448038 | controller | - ' logger.go:42: 11:37:51 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.448044 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.448050 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.448056 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.448062 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.448068 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.448083 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.448090 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.448100 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.448106 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.448112 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.448126 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.448132 | controller | == 1 ]' 2026-01-21 11:38:24.448138 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.448144 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.448150 | controller | - ' else' 2026-01-21 11:38:24.448156 | controller | - ' exit 1' 2026-01-21 11:38:24.448162 | controller | - ' fi' 2026-01-21 11:38:24.448168 | controller | - ' ]' 2026-01-21 11:38:24.448174 | controller | - ' logger.go:42: 11:37:51 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.448180 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.448186 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.448192 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.448198 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.448203 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.448209 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.448215 | controller | -1' 2026-01-21 11:38:24.448221 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.448227 | controller | -v ''^$''' 2026-01-21 11:38:24.448233 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.448252 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448258 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.448264 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.448270 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.448275 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.448281 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.448287 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.448293 | controller | ''#'' Global config' 2026-01-21 11:38:24.448299 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448305 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.448311 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.448317 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.448331 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.448338 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.448344 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.448350 | controller | ''#'' Service config' 2026-01-21 11:38:24.448356 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448362 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.448368 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.448374 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.448384 | controller | - ' logger.go:42: 11:37:52 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.448390 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.448396 | controller | - ' logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.448402 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.448407 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.448413 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.448419 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.448425 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.448431 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.448437 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.448443 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.448449 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.448455 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.448461 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.448467 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.448473 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.448479 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.448484 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.448490 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.448496 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.448502 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.448508 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.448514 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.448520 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.448526 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.448532 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.448537 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.448543 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.448549 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.448555 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.448573 | controller | period = 900' 2026-01-21 11:38:24.448579 | controller | - ' logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448585 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.448594 | controller | - ' logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.448600 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.448606 | controller | - ' logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.448612 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.448627 | controller | - ' logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.448633 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.448645 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.448651 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.448657 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.448663 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.448669 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.448675 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.448688 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.448695 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.448701 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.448706 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.448736 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.448742 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.448748 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.448754 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.448760 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.448766 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.448772 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.448778 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.448784 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.448789 | controller | - ' logger.go:42: 11:37:53 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448795 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.448801 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.448807 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.448813 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.448819 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.448825 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.448835 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.448841 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.448847 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.448853 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.448858 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.448864 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.448870 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.448876 | controller | == 1 ]' 2026-01-21 11:38:24.448882 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.448888 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.448894 | controller | - ' else' 2026-01-21 11:38:24.448900 | controller | - ' exit 1' 2026-01-21 11:38:24.448906 | controller | - ' fi' 2026-01-21 11:38:24.448912 | controller | - ' ]' 2026-01-21 11:38:24.448918 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.448924 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.448930 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.448936 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.448942 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.448954 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.448960 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.448966 | controller | -1' 2026-01-21 11:38:24.448972 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.448978 | controller | -v ''^$''' 2026-01-21 11:38:24.448984 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.448990 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.448996 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.449002 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449008 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.449014 | controller | - ' logger.go:42: 11:37:54 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449020 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.449026 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449032 | controller | ''#'' Global config' 2026-01-21 11:38:24.449042 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449048 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.449054 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449060 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.449066 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449072 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.449078 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449083 | controller | ''#'' Service config' 2026-01-21 11:38:24.449089 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449106 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.449112 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449118 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.449124 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449130 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.449136 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449141 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.449147 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.449153 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.449159 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.449165 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.449171 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.449177 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.449184 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.449191 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.449197 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.449203 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.449209 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.449216 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.449222 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.449228 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.449234 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.449239 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.449249 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.449255 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.449261 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.449267 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.449273 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.449280 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.449287 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.449299 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.449306 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.449314 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.449321 | controller | period = 900' 2026-01-21 11:38:24.449328 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449335 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.449342 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449348 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.449354 | controller | - ' logger.go:42: 11:37:55 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449360 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.449366 | controller | - ' logger.go:42: 11:37:56 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449372 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.449383 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.449389 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.449395 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.449401 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.449407 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.449412 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.449418 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.449424 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.449430 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.449436 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.449451 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.449457 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.449463 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.449468 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.449474 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.449480 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.449486 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.449492 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.449498 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.449503 | controller | - ' logger.go:42: 11:37:56 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449509 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.449515 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.449527 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.449534 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.449542 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.449549 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.449557 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.449564 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.449577 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.449586 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.449593 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.449601 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.449608 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.449615 | controller | == 1 ]' 2026-01-21 11:38:24.449667 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.449676 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.449682 | controller | - ' else' 2026-01-21 11:38:24.449688 | controller | - ' exit 1' 2026-01-21 11:38:24.449694 | controller | - ' fi' 2026-01-21 11:38:24.449700 | controller | - ' ]' 2026-01-21 11:38:24.449706 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.449730 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.449739 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.449745 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.449751 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.449757 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.449763 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449768 | controller | -v ''^$''' 2026-01-21 11:38:24.449774 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.449780 | controller | -1' 2026-01-21 11:38:24.449786 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.449792 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449798 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.449804 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449810 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.449816 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449821 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.449827 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449833 | controller | ''#'' Global config' 2026-01-21 11:38:24.449839 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449845 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.449851 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449857 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.449863 | controller | - ' logger.go:42: 11:37:57 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449868 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.449879 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449885 | controller | ''#'' Service config' 2026-01-21 11:38:24.449891 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.449897 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.449902 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.449908 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.449914 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.449920 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.449926 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.449954 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.449962 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.449968 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.449974 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.449980 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.449986 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.450006 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.450013 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.450019 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.450025 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.450031 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.450037 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.450042 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.450048 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.450054 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.450060 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.450066 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.450072 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.450078 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.450083 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.450089 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.450108 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.450115 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.450120 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.450126 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.450132 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.450138 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.450144 | controller | period = 900' 2026-01-21 11:38:24.450150 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.450155 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.450161 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.450167 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.450173 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.450179 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.450185 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.450191 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.450202 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.450208 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.450214 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.450220 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.450226 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.450232 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.450239 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.450246 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.450252 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.450258 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.450272 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.450278 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.450290 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.450296 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.450302 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.450308 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.450314 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.450319 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.450325 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.450335 | controller | - ' logger.go:42: 11:37:58 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.450341 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.450351 | controller | - ' logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.450365 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.450371 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.450377 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.450383 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.450389 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.450395 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.450401 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.450407 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.450413 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.450419 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.450425 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.450431 | controller | == 1 ]' 2026-01-21 11:38:24.450437 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.450443 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.450449 | controller | - ' else' 2026-01-21 11:38:24.450455 | controller | - ' exit 1' 2026-01-21 11:38:24.450463 | controller | - ' fi' 2026-01-21 11:38:24.450481 | controller | - ' ]' 2026-01-21 11:38:24.450492 | controller | - ' logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.450501 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.450512 | controller | - ' logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.450521 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.450530 | controller | - ' logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.450538 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.450547 | controller | - ' logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.450556 | controller | -v ''^$''' 2026-01-21 11:38:24.450564 | controller | - ' logger.go:42: 11:37:59 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.450573 | controller | -1' 2026-01-21 11:38:24.450582 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.450611 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.450621 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.450628 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.450635 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.450642 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.450650 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.450658 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.450665 | controller | ''#'' Global config' 2026-01-21 11:38:24.450671 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.450677 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.450683 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.450694 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.450700 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.450706 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.450737 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.450748 | controller | ''#'' Service config' 2026-01-21 11:38:24.450754 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.450760 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.450766 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.450779 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.450785 | controller | - ' logger.go:42: 11:38:00 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.450791 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.450797 | controller | - ' logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.450803 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.450809 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.450815 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.450821 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.450832 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.450839 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.450845 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.450851 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.450857 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.450863 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.450869 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.450875 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.450881 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.450886 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.450892 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.450898 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.450904 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.450910 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.450916 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.450950 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.450957 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.450963 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.450968 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.450974 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.450980 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.450986 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.450992 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.450998 | controller | period = 900' 2026-01-21 11:38:24.451014 | controller | - ' logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451021 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.451027 | controller | - ' logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.451033 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.451039 | controller | - ' logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.451045 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.451051 | controller | - ' logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.451056 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.451073 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.451079 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.451085 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.451091 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.451097 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.451103 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.451108 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.451114 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.451120 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.451126 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.451138 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.451144 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.451149 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.451155 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.451161 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.451167 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.451177 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.451183 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.451189 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.451195 | controller | - ' logger.go:42: 11:38:01 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451201 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.451207 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.451212 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.451218 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.451224 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.451230 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.451236 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.451242 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.451248 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.451254 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.451260 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.451266 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.451272 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.451278 | controller | == 1 ]' 2026-01-21 11:38:24.451283 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.451289 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.451295 | controller | - ' else' 2026-01-21 11:38:24.451301 | controller | - ' exit 1' 2026-01-21 11:38:24.451307 | controller | - ' fi' 2026-01-21 11:38:24.451313 | controller | - ' ]' 2026-01-21 11:38:24.451319 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.451325 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.451331 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.451337 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.451346 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.451352 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.451358 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.451364 | controller | -1' 2026-01-21 11:38:24.451370 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.451376 | controller | -v ''^$''' 2026-01-21 11:38:24.451382 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.451390 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451396 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.451402 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.451412 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.451418 | controller | - ' logger.go:42: 11:38:02 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.451424 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.451430 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.451440 | controller | ''#'' Global config' 2026-01-21 11:38:24.451446 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451452 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.451458 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.451464 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.451470 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.451476 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.451482 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.451488 | controller | ''#'' Service config' 2026-01-21 11:38:24.451494 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451500 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.451505 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.451511 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.451517 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.451523 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.451529 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.451535 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.451541 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.451547 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.451553 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.451559 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.451565 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.451571 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.451577 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.451583 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.451589 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.451595 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.451600 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.451606 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.451612 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.451618 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.451626 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.451634 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.451646 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.451654 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.451667 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.451677 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.451686 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.451694 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.451708 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.451745 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.451754 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.451761 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.451768 | controller | period = 900' 2026-01-21 11:38:24.451776 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451783 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.451790 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.451796 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.451802 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.451808 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.451814 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.451820 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.451832 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.451837 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.451843 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.451854 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.451860 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.451866 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.451872 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.451878 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.451884 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.451890 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.451902 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.451907 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.451913 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.451927 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.451933 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.451939 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.451945 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.451951 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.451956 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.451962 | controller | - ' logger.go:42: 11:38:03 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.451968 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.451974 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.451980 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.451986 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.451992 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.451998 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.452004 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.452010 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.452016 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.452022 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.452027 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.452033 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.452040 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.452046 | controller | == 1 ]' 2026-01-21 11:38:24.452057 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.452063 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.452069 | controller | - ' else' 2026-01-21 11:38:24.452075 | controller | - ' exit 1' 2026-01-21 11:38:24.452081 | controller | - ' fi' 2026-01-21 11:38:24.452087 | controller | - ' ]' 2026-01-21 11:38:24.452093 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.452099 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.452104 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.452110 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.452116 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.452122 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.452128 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.452134 | controller | -v ''^$''' 2026-01-21 11:38:24.452140 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.452146 | controller | -1' 2026-01-21 11:38:24.452152 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.452158 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.452164 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.452170 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.452176 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.452197 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.452203 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.452209 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.452215 | controller | ''#'' Global config' 2026-01-21 11:38:24.452221 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.452227 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.452233 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.452238 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.452244 | controller | - ' logger.go:42: 11:38:05 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.452250 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.452256 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.452262 | controller | ''#'' Service config' 2026-01-21 11:38:24.452268 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.452274 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.452283 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.452289 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.452295 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.452301 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.452307 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.452313 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.452322 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.452328 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.452335 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.452341 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.452347 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.452353 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.452359 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.452365 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.452371 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.452377 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.452387 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.452394 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.452400 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.452445 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.452452 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.452458 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.452464 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.452470 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.452477 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.452483 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.452489 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.452495 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.452501 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.452506 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.452512 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.452518 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.452524 | controller | period = 900' 2026-01-21 11:38:24.452530 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.452536 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.452542 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.452548 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.452554 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.452560 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.452566 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.452572 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.452584 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.452590 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.452596 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.452601 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.452607 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.452613 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.452619 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.452625 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.452631 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.452641 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.452653 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.452663 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.452669 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.452675 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.452683 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.452691 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.452699 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.452707 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.452746 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.452771 | controller | - ' logger.go:42: 11:38:06 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.452780 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.452788 | controller | - ' logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | running 2026-01-21 11:38:24.452795 | controller | command: [sh -c set -euxo pipefail' 2026-01-21 11:38:24.452802 | controller | - ' oc project watcher-kuttl-default' 2026-01-21 11:38:24.452811 | controller | - ' APIPOD=$(oc get pods -n watcher-kuttl-default -l "service=watcher-api" 2026-01-21 11:38:24.452818 | controller | -ocustom-columns=:metadata.name|grep -v ^$|head -1)' 2026-01-21 11:38:24.452826 | controller | - ' if [ -n "${APIPOD}" ]; then' 2026-01-21 11:38:24.452834 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/01-global-custom.conf) 2026-01-21 11:38:24.452840 | controller | |grep -c "^# Global config") == 1 ]' 2026-01-21 11:38:24.452846 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/02-service-custom.conf) 2026-01-21 11:38:24.452852 | controller | |grep -c "^# Service config") == 1 ]' 2026-01-21 11:38:24.452858 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/watcher/watcher.conf.d/00-default.conf) 2026-01-21 11:38:24.452869 | controller | |grep -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem'') 2026-01-21 11:38:24.452875 | controller | == 1 ]' 2026-01-21 11:38:24.452881 | controller | - ' [ $(echo $(oc rsh -c watcher-api ${APIPOD} cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf) 2026-01-21 11:38:24.452887 | controller | |grep -czPo ''TimeOut 80'') == 1 ]' 2026-01-21 11:38:24.452893 | controller | - ' else' 2026-01-21 11:38:24.452899 | controller | - ' exit 1' 2026-01-21 11:38:24.452905 | controller | - ' fi' 2026-01-21 11:38:24.452911 | controller | - ' ]' 2026-01-21 11:38:24.452917 | controller | - ' logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | + oc project 2026-01-21 11:38:24.452922 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.452928 | controller | - ' logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | Already 2026-01-21 11:38:24.452934 | controller | on project "watcher-kuttl-default" on server "https://api.crc.testing:6443".' 2026-01-21 11:38:24.452940 | controller | - ' logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | ++ oc get 2026-01-21 11:38:24.452946 | controller | pods -n watcher-kuttl-default -l service=watcher-api -ocustom-columns=:metadata.name' 2026-01-21 11:38:24.452952 | controller | - ' logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.452958 | controller | -v ''^$''' 2026-01-21 11:38:24.452963 | controller | - ' logger.go:42: 11:38:07 | watcher/4-deploy-with-precreated-account | ++ head 2026-01-21 11:38:24.452969 | controller | -1' 2026-01-21 11:38:24.452975 | controller | - ' logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account | + APIPOD=watcher-kuttl-api-0' 2026-01-21 11:38:24.452985 | controller | - ' logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.452991 | controller | -n watcher-kuttl-api-0 '']''' 2026-01-21 11:38:24.452997 | controller | - ' logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.453003 | controller | -c ''^# Global config''' 2026-01-21 11:38:24.453009 | controller | - ' logger.go:42: 11:38:08 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.453015 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/01-global-custom.conf' 2026-01-21 11:38:24.453020 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.453026 | controller | ''#'' Global config' 2026-01-21 11:38:24.453032 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.453038 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.453044 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.453050 | controller | -c ''^# Service config''' 2026-01-21 11:38:24.453056 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.453062 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/02-service-custom.conf' 2026-01-21 11:38:24.453067 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.453073 | controller | ''#'' Service config' 2026-01-21 11:38:24.453079 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.453085 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.453091 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.453097 | controller | -czPo ''\[prometheus_client\]\s+host\s+=\s+metric-storage-prometheus.watcher-kuttl-default.svc\s+port\s+=\s+9090\s+cafile\s+=\s+/etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem''' 2026-01-21 11:38:24.453102 | controller | - ' logger.go:42: 11:38:09 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.453108 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/watcher/watcher.conf.d/00-default.conf' 2026-01-21 11:38:24.453114 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.453120 | controller | ''[DEFAULT]'' state_path = /var/lib/watcher transport_url = ''rabbit://default_user_aVigJBAvrYhByMd-Ul2:EGVyYRhtxRad-dzxzqRpf5RuhWpAg44t@rabbitmq.watcher-kuttl-default.svc:5671/?ssl=1'' 2026-01-21 11:38:24.453130 | controller | control_exchange = watcher debug = True log_file = /var/log/watcher/watcher-kuttl-api.log 2026-01-21 11:38:24.453136 | controller | ''#'' empty notification_level means that no notification will be sent notification_level 2026-01-21 11:38:24.453142 | controller | = ''[database]'' connection = ''mysql+pymysql://watcher_test:c9d9db3f978dbad93c9a1b57e07b83c8@openstack.watcher-kuttl-default.svc/watcher?read_default_file=/etc/my.cnf'' 2026-01-21 11:38:24.453149 | controller | ''[oslo_policy]'' policy_file = /etc/watcher/policy.yaml.sample ''[oslo_messaging_notifications]'' 2026-01-21 11:38:24.453155 | controller | driver = noop ''[oslo_messaging_rabbit]'' rabbit_quorum_queue=true rabbit_transient_quorum_queue=true 2026-01-21 11:38:24.453161 | controller | amqp_durable_queues=true ''[keystone_authtoken]'' memcached_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.453166 | controller | tls_enabled=true memcache_tls_certfile = /etc/pki/tls/certs/mtls.crt memcache_tls_keyfile 2026-01-21 11:38:24.453172 | controller | = /etc/pki/tls/private/mtls.key memcache_tls_cafile = /etc/pki/tls/certs/mtls-ca.crt 2026-01-21 11:38:24.453178 | controller | memcache_tls_enabled = true project_domain_name = Default project_name = service 2026-01-21 11:38:24.453184 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.453190 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.453209 | controller | ''[watcher_clients_auth]'' project_domain_name = Default project_name = service 2026-01-21 11:38:24.453215 | controller | user_domain_name = Default password = password username = watcher auth_url = https://keystone-internal.watcher-kuttl-default.svc:5000 2026-01-21 11:38:24.453221 | controller | interface = internal auth_type = password cafile = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem 2026-01-21 11:38:24.453227 | controller | ''[oslo_concurrency]'' lock_path = /var/lib/watcher/tmp ''[watcher_datasources]'' 2026-01-21 11:38:24.453235 | controller | datasources = prometheus ''[cache]'' backend = oslo_cache.memcache_pool memcache_servers=memcached-0.memcached.watcher-kuttl-default.svc:11212 2026-01-21 11:38:24.453242 | controller | memcache_socket_timeout = 0.5 memcache_pool_connection_get_timeout = 1 enabled=true 2026-01-21 11:38:24.453248 | controller | tls_enabled=true tls_certfile=/etc/pki/tls/certs/mtls.crt tls_keyfile=/etc/pki/tls/private/mtls.key 2026-01-21 11:38:24.453254 | controller | tls_cafile=/etc/pki/tls/certs/mtls-ca.crt memcache_dead_retry = 30 ''[prometheus_client]'' 2026-01-21 11:38:24.453271 | controller | host = metric-storage-prometheus.watcher-kuttl-default.svc port = 9090 cafile = 2026-01-21 11:38:24.453277 | controller | /etc/pki/ca-trust/extracted/pem/prometheus/internal-ca-bundle.pem ''[cinder_client]'' 2026-01-21 11:38:24.453285 | controller | endpoint_type = internal ''[glance_client]'' endpoint_type = internal ''[ironic_client]'' 2026-01-21 11:38:24.453291 | controller | endpoint_type = internal ''[keystone_client]'' interface = internal ''[neutron_client]'' 2026-01-21 11:38:24.453297 | controller | endpoint_type = internal ''[nova_client]'' endpoint_type = internal ''[placement_client]'' 2026-01-21 11:38:24.453303 | controller | interface = internal ''[watcher_cluster_data_model_collectors.compute]'' period 2026-01-21 11:38:24.453309 | controller | = 900 ''[watcher_cluster_data_model_collectors.baremetal]'' period = 900 ''[watcher_cluster_data_model_collectors.storage]'' 2026-01-21 11:38:24.453315 | controller | period = 900' 2026-01-21 11:38:24.453321 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.453326 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.453332 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | ++ grep 2026-01-21 11:38:24.453338 | controller | -czPo ''TimeOut 80''' 2026-01-21 11:38:24.453344 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | +++ oc 2026-01-21 11:38:24.453350 | controller | rsh -c watcher-api watcher-kuttl-api-0 cat /etc/httpd/conf.d/10-watcher-wsgi-main.conf' 2026-01-21 11:38:24.453356 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | ++ echo 2026-01-21 11:38:24.453362 | controller | ''#'' internal vhost watcher-internal.watcher-kuttl-default.svc configuration '''' ServerName watcher-internal.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.453374 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.453380 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.453385 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.453391 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.453397 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.453403 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.453409 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.453421 | controller | internal display-name=internal group=watcher processes=2 threads=1 user=watcher 2026-01-21 11:38:24.453427 | controller | WSGIProcessGroup internal WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' '''' 2026-01-21 11:38:24.453452 | controller | ''#'' public vhost watcher-public.watcher-kuttl-default.svc configuration '''' ServerName watcher-public.watcher-kuttl-default.svc ''##'' Vhost docroot 2026-01-21 11:38:24.453475 | controller | DocumentRoot ''"/var/www/cgi-bin"'' ''#'' Set the timeout for the watcher-api TimeOut 2026-01-21 11:38:24.453482 | controller | 80 ''##'' Directories, there should at least be a declaration for /var/www/cgi-bin 2026-01-21 11:38:24.453490 | controller | '''' Options -Indexes +FollowSymLinks +MultiViews 2026-01-21 11:38:24.453497 | controller | AllowOverride None Require all granted '''' ''##'' Logging ErrorLog 2026-01-21 11:38:24.453504 | controller | /dev/stdout ServerSignature Off CustomLog /dev/stdout combined ''env=!forwarded'' 2026-01-21 11:38:24.453511 | controller | CustomLog /dev/stdout proxy env=forwarded ''##'' set watcher log level to debug 2026-01-21 11:38:24.453519 | controller | LogLevel debug ''##'' WSGI configuration WSGIApplicationGroup ''%{GLOBAL}'' WSGIDaemonProcess 2026-01-21 11:38:24.453526 | controller | public display-name=public group=watcher processes=2 threads=1 user=watcher WSGIProcessGroup 2026-01-21 11:38:24.453533 | controller | public WSGIScriptAlias / ''"/usr/bin/watcher-api-wsgi"'' ''''' 2026-01-21 11:38:24.453540 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | + ''['' 2026-01-21 11:38:24.453548 | controller | 1 == 1 '']''' 2026-01-21 11:38:24.453555 | controller | - ' logger.go:42: 11:38:10 | watcher/4-deploy-with-precreated-account | test step 2026-01-21 11:38:24.453563 | controller | completed 4-deploy-with-precreated-account' 2026-01-21 11:38:24.453570 | controller | - ' logger.go:42: 11:38:10 | watcher/5-cleanup-watcher | starting test step 5-cleanup-watcher' 2026-01-21 11:38:24.453576 | controller | - ' logger.go:42: 11:38:10 | watcher/5-cleanup-watcher | running command: [sh -c 2026-01-21 11:38:24.453582 | controller | set -ex' 2026-01-21 11:38:24.453588 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.453594 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.453599 | controller | - ' ]' 2026-01-21 11:38:24.453605 | controller | - ' logger.go:42: 11:38:10 | watcher/5-cleanup-watcher | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.453611 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.453617 | controller | - ' logger.go:42: 11:38:10 | watcher/5-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.453623 | controller | - ' logger.go:42: 11:38:15 | watcher/5-cleanup-watcher | + ''['' 1 == 0 '']''' 2026-01-21 11:38:24.453634 | controller | - ' logger.go:42: 11:38:16 | watcher/5-cleanup-watcher | running command: [sh -c 2026-01-21 11:38:24.453640 | controller | set -ex' 2026-01-21 11:38:24.453646 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.453652 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.453658 | controller | - ' ]' 2026-01-21 11:38:24.453664 | controller | - ' logger.go:42: 11:38:16 | watcher/5-cleanup-watcher | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.453670 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.453679 | controller | - ' logger.go:42: 11:38:16 | watcher/5-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.453685 | controller | - ' logger.go:42: 11:38:19 | watcher/5-cleanup-watcher | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.453691 | controller | - ' logger.go:42: 11:38:20 | watcher/5-cleanup-watcher | running command: [sh -c 2026-01-21 11:38:24.453697 | controller | set -ex' 2026-01-21 11:38:24.453703 | controller | - ' oc exec -n watcher-kuttl-default openstackclient -- openstack service list 2026-01-21 11:38:24.453727 | controller | -f value -c Name -c Type | [ $(grep -c ^watcher) == 0 ]' 2026-01-21 11:38:24.453740 | controller | - ' ]' 2026-01-21 11:38:24.453748 | controller | - ' logger.go:42: 11:38:20 | watcher/5-cleanup-watcher | + oc exec -n watcher-kuttl-default 2026-01-21 11:38:24.453780 | controller | openstackclient -- openstack service list -f value -c Name -c Type' 2026-01-21 11:38:24.453788 | controller | - ' logger.go:42: 11:38:20 | watcher/5-cleanup-watcher | ++ grep -c ''^watcher''' 2026-01-21 11:38:24.453794 | controller | - ' logger.go:42: 11:38:23 | watcher/5-cleanup-watcher | + ''['' 0 == 0 '']''' 2026-01-21 11:38:24.453800 | controller | - ' logger.go:42: 11:38:23 | watcher/5-cleanup-watcher | test step completed 5-cleanup-watcher' 2026-01-21 11:38:24.453806 | controller | - ' logger.go:42: 11:38:23 | watcher | skipping kubernetes event logging' 2026-01-21 11:38:24.453811 | controller | - === CONT kuttl/harness/deps 2026-01-21 11:38:24.453817 | controller | - ' logger.go:42: 11:38:24 | deps | Ignoring infra.yaml as it does not match file 2026-01-21 11:38:24.453823 | controller | name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.453829 | controller | - ' logger.go:42: 11:38:24 | deps | Ignoring keystone.yaml as it does not match 2026-01-21 11:38:24.453835 | controller | file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.453841 | controller | - ' logger.go:42: 11:38:24 | deps | Ignoring kustomization.yaml as it does not 2026-01-21 11:38:24.453847 | controller | match file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.453859 | controller | - ' logger.go:42: 11:38:24 | deps | Ignoring namespace.yaml as it does not match 2026-01-21 11:38:24.479142 | controller | file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.479196 | controller | - ' logger.go:42: 11:38:24 | deps | Ignoring telemetry.yaml as it does not match 2026-01-21 11:38:24.479206 | controller | file name regexp: ^(\d+)-(?:[^\.]+)(?:\.yaml)?$' 2026-01-21 11:38:24.479212 | controller | - ' logger.go:42: 11:38:24 | deps | Skipping creation of user-supplied namespace: 2026-01-21 11:38:24.479218 | controller | watcher-kuttl-default' 2026-01-21 11:38:24.479225 | controller | - ' logger.go:42: 11:38:24 | deps | skipping kubernetes event logging' 2026-01-21 11:38:24.479232 | controller | - === NAME kuttl 2026-01-21 11:38:24.479240 | controller | - ' harness.go:406: run tests finished' 2026-01-21 11:38:24.479255 | controller | - ' harness.go:514: cleaning up' 2026-01-21 11:38:24.479261 | controller | - ' harness.go:571: removing temp folder: ""' 2026-01-21 11:38:24.479269 | controller | - '--- FAIL: kuttl (1017.99s)' 2026-01-21 11:38:24.479276 | controller | - ' --- FAIL: kuttl/harness (0.00s)' 2026-01-21 11:38:24.479283 | controller | - ' --- PASS: kuttl/harness/common (0.01s)' 2026-01-21 11:38:24.479290 | controller | - ' --- PASS: kuttl/harness/watcher-topology (80.92s)' 2026-01-21 11:38:24.479297 | controller | - ' --- PASS: kuttl/harness/watcher-tls-certs-change (51.51s)' 2026-01-21 11:38:24.479303 | controller | - ' --- PASS: kuttl/harness/watcher-tls (159.28s)' 2026-01-21 11:38:24.479309 | controller | - ' --- FAIL: kuttl/harness/watcher-rmquser (300.24s)' 2026-01-21 11:38:24.479315 | controller | - ' --- PASS: kuttl/harness/watcher-notification (58.53s)' 2026-01-21 11:38:24.479321 | controller | - ' --- PASS: kuttl/harness/watcher-cinder (179.74s)' 2026-01-21 11:38:24.479327 | controller | - ' --- PASS: kuttl/harness/watcher-api-scaling (75.82s)' 2026-01-21 11:38:24.479333 | controller | - ' --- PASS: kuttl/harness/watcher (111.88s)' 2026-01-21 11:38:24.479339 | controller | - ' --- PASS: kuttl/harness/deps (0.01s)' 2026-01-21 11:38:24.479345 | controller | - FAIL 2026-01-21 11:38:24.479360 | controller | 2026-01-21 11:38:24.612961 | controller | NO MORE HOSTS LEFT ************************************************************* 2026-01-21 11:38:24.613109 | controller | 2026-01-21 11:38:24.613235 | controller | PLAY RECAP ********************************************************************* 2026-01-21 11:38:24.613246 | controller | controller : ok=2 changed=0 unreachable=0 failed=1 skipped=1 rescued=0 ignored=0 2026-01-21 11:38:24.613255 | controller | 2026-01-21 11:38:24.613262 | controller | Wednesday 21 January 2026 11:38:24 +0000 (0:16:59.262) 0:16:59.918 ***** 2026-01-21 11:38:24.613270 | controller | =============================================================================== 2026-01-21 11:38:24.613279 | controller | run kuttl test suite from operator Makefile -------------------------- 1019.26s 2026-01-21 11:38:24.613302 | controller | run_hook : Loop on hooks for pre_kuttl_from_operator -------------------- 0.26s 2026-01-21 11:38:24.613311 | controller | run_hook : Assert single hooks are all mappings ------------------------- 0.14s 2026-01-21 11:38:24.613319 | controller | run_hook : Assert parameters are valid ---------------------------------- 0.12s 2026-01-21 11:38:24.613327 | controller | Run hooks before running kuttl tests ------------------------------------ 0.07s 2026-01-21 11:38:24.613335 | controller | Wednesday 21 January 2026 11:38:24 +0000 (0:16:59.262) 0:16:59.916 ***** 2026-01-21 11:38:24.613343 | controller | =============================================================================== 2026-01-21 11:38:24.613351 | controller | ansible.builtin.command ---------------------------------------------- 1019.26s 2026-01-21 11:38:24.613359 | controller | run_hook ---------------------------------------------------------------- 0.53s 2026-01-21 11:38:24.613367 | controller | ansible.builtin.include_role -------------------------------------------- 0.07s 2026-01-21 11:38:24.613375 | controller | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2026-01-21 11:38:24.613383 | controller | total ---------------------------------------------------------------- 1019.86s 2026-01-21 11:38:25.051329 | controller | ERROR 2026-01-21 11:38:25.051642 | controller | { 2026-01-21 11:38:25.051750 | controller | "delta": "0:17:00.799932", 2026-01-21 11:38:25.051800 | controller | "end": "2026-01-21 11:38:24.618520", 2026-01-21 11:38:25.051897 | controller | "msg": "non-zero return code", 2026-01-21 11:38:25.051959 | controller | "rc": 2, 2026-01-21 11:38:25.051996 | controller | "start": "2026-01-21 11:21:23.818588" 2026-01-21 11:38:25.052031 | controller | } failure 2026-01-21 11:38:25.236513 | 2026-01-21 11:38:25.236642 | PLAY RECAP 2026-01-21 11:38:25.236708 | controller | ok: 0 changed: 0 unreachable: 0 failed: 1 skipped: 0 rescued: 0 ignored: 0 2026-01-21 11:38:25.236735 | 2026-01-21 11:38:25.352447 | RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/kuttl/kuttl-from-operator-run.yaml@main] 2026-01-21 11:38:25.367801 | POST-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/e2e-collect-logs.yml@main] 2026-01-21 11:38:26.066957 | 2026-01-21 11:38:26.067095 | PLAY [Run ci/playbooks/e2e-collect-logs.yml] 2026-01-21 11:38:26.113290 | 2026-01-21 11:38:26.113439 | TASK [Filter out host if needed] 2026-01-21 11:38:26.125671 | controller | skipping: end_host conditional evaluated to False, continuing execution for controller 2026-01-21 11:38:26.132247 | 2026-01-21 11:38:26.132315 | TASK [Filter out host if needed] 2026-01-21 11:38:26.148410 | 2026-01-21 11:38:26.148520 | TASK [Ensure we have the ci-framework on host] 2026-01-21 11:38:26.809384 | controller | ok 2026-01-21 11:38:26.823906 | 2026-01-21 11:38:26.824053 | TASK [End host if no basedir] 2026-01-21 11:38:26.839741 | controller | skipping: end_host conditional evaluated to False, continuing execution for controller 2026-01-21 11:38:26.850291 | 2026-01-21 11:38:26.850373 | TASK [Read base centos-9 scenarios] 2026-01-21 11:38:26.874929 | controller | ok 2026-01-21 11:38:26.975073 | 2026-01-21 11:38:26.975214 | TASK [cifmw_helpers : Fail if file is not yaml or yml extension] 2026-01-21 11:38:27.009895 | controller | skipping: Conditional result was False 2026-01-21 11:38:27.020380 | 2026-01-21 11:38:27.020495 | TASK [cifmw_helpers : Check if file is available] 2026-01-21 11:38:27.346271 | controller | ok 2026-01-21 11:38:27.355470 | 2026-01-21 11:38:27.355577 | TASK [cifmw_helpers : Create temporary directory] 2026-01-21 11:38:27.391977 | controller | skipping: Conditional result was False 2026-01-21 11:38:27.403264 | 2026-01-21 11:38:27.403366 | TASK [cifmw_helpers : Read vars] 2026-01-21 11:38:27.445365 | 2026-01-21 11:38:27.445637 | TASK [cifmw_helpers : Create new variable files with content] 2026-01-21 11:38:27.492931 | 2026-01-21 11:38:27.493136 | TASK [cifmw_helpers : Include vars] 2026-01-21 11:38:27.538825 | 2026-01-21 11:38:27.539053 | TASK [cifmw_helpers : Remove temporary directory] 2026-01-21 11:38:27.565029 | controller | skipping: Conditional result was False 2026-01-21 11:38:27.586181 | 2026-01-21 11:38:27.586316 | TASK [run_hook : Assert parameters are valid] 2026-01-21 11:38:27.686731 | controller | ok: All assertions passed 2026-01-21 11:38:27.699895 | 2026-01-21 11:38:27.700047 | TASK [run_hook : Assert single hooks are all mappings] 2026-01-21 11:38:27.793434 | controller | ok: All assertions passed 2026-01-21 11:38:27.810853 | 2026-01-21 11:38:27.811019 | LOOP [run_hook : Loop on hooks for pre_logs] 2026-01-21 11:38:27.918346 | 2026-01-21 11:38:27.918630 | TASK [cifmw_setup : Ensure cifmw_basedir param is set] 2026-01-21 11:38:27.969300 | controller | ok 2026-01-21 11:38:27.982783 | 2026-01-21 11:38:27.982917 | TASK [Read artifacts parameters dir and set as facts] 2026-01-21 11:38:28.016295 | controller | ok 2026-01-21 11:38:28.044488 | 2026-01-21 11:38:28.044589 | TASK [cifmw_helpers : Check directory is available] 2026-01-21 11:38:28.336835 | controller | ok 2026-01-21 11:38:28.350472 | 2026-01-21 11:38:28.350611 | TASK [cifmw_helpers : Find yaml files] 2026-01-21 11:38:28.719942 | controller | ok: All paths examined 2026-01-21 11:38:28.737108 | 2026-01-21 11:38:28.737255 | TASK [cifmw_helpers : Print available yaml files] 2026-01-21 11:38:28.798364 | controller | ok: Found yaml files to parse: ['/home/zuul/ci-framework-data/artifacts/parameters/zuul-params.yml', '/home/zuul/ci-framework-data/artifacts/parameters/install-yamls-params.yml', '/home/zuul/ci-framework-data/artifacts/parameters/custom-params.yml', '/home/zuul/ci-framework-data/artifacts/parameters/openshift-login-params.yml'] 2026-01-21 11:38:28.812265 | 2026-01-21 11:38:28.812399 | LOOP [cifmw_helpers : Create files on localhost and use include_vars] 2026-01-21 11:38:28.863991 | controller | Output suppressed because no_log was given 2026-01-21 11:38:28.864326 | 2026-01-21 11:38:28.872651 | controller | Output suppressed because no_log was given 2026-01-21 11:38:28.880206 | controller | Output suppressed because no_log was given 2026-01-21 11:38:28.887529 | controller | Output suppressed because no_log was given 2026-01-21 11:38:28.925277 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/untrusted/project_0/github.com/openstack-k8s-operators/ci-framework/roles/cifmw_helpers/tasks/include_file.yml 2026-01-21 11:38:28.936417 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/untrusted/project_0/github.com/openstack-k8s-operators/ci-framework/roles/cifmw_helpers/tasks/include_file.yml 2026-01-21 11:38:28.944620 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/untrusted/project_0/github.com/openstack-k8s-operators/ci-framework/roles/cifmw_helpers/tasks/include_file.yml 2026-01-21 11:38:28.951228 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/untrusted/project_0/github.com/openstack-k8s-operators/ci-framework/roles/cifmw_helpers/tasks/include_file.yml 2026-01-21 11:38:28.963375 | 2026-01-21 11:38:28.963470 | TASK [cifmw_helpers : Fail if file is not yaml or yml extension] 2026-01-21 11:38:28.999746 | controller | skipping: Conditional result was False 2026-01-21 11:38:29.016129 | 2026-01-21 11:38:29.016275 | TASK [cifmw_helpers : Check if file is available] 2026-01-21 11:38:29.319409 | controller | ok 2026-01-21 11:38:29.328741 | 2026-01-21 11:38:29.328876 | TASK [cifmw_helpers : Create temporary directory] 2026-01-21 11:38:29.793334 | controller -> localhost | changed 2026-01-21 11:38:29.807291 | 2026-01-21 11:38:29.807460 | TASK [cifmw_helpers : Read vars] 2026-01-21 11:38:30.224285 | controller | Output suppressed because no_log was given 2026-01-21 11:38:30.237562 | 2026-01-21 11:38:30.237753 | TASK [cifmw_helpers : Create new variable files with content] 2026-01-21 11:38:30.907881 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:30.915164 | 2026-01-21 11:38:30.915238 | TASK [cifmw_helpers : Include vars] 2026-01-21 11:38:30.972786 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:30.988561 | 2026-01-21 11:38:30.988753 | TASK [cifmw_helpers : Remove temporary directory] 2026-01-21 11:38:31.570089 | controller | ok 2026-01-21 11:38:31.584780 | 2026-01-21 11:38:31.584928 | TASK [cifmw_helpers : Fail if file is not yaml or yml extension] 2026-01-21 11:38:31.624000 | controller | skipping: Conditional result was False 2026-01-21 11:38:31.640235 | 2026-01-21 11:38:31.640463 | TASK [cifmw_helpers : Check if file is available] 2026-01-21 11:38:31.959993 | controller | ok 2026-01-21 11:38:31.975181 | 2026-01-21 11:38:31.975325 | TASK [cifmw_helpers : Create temporary directory] 2026-01-21 11:38:32.301003 | controller -> localhost | changed 2026-01-21 11:38:32.316759 | 2026-01-21 11:38:32.316897 | TASK [cifmw_helpers : Read vars] 2026-01-21 11:38:32.581606 | controller | Output suppressed because no_log was given 2026-01-21 11:38:32.595389 | 2026-01-21 11:38:32.595533 | TASK [cifmw_helpers : Create new variable files with content] 2026-01-21 11:38:33.111095 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:33.118915 | 2026-01-21 11:38:33.119025 | TASK [cifmw_helpers : Include vars] 2026-01-21 11:38:33.183730 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:33.191405 | 2026-01-21 11:38:33.191501 | TASK [cifmw_helpers : Remove temporary directory] 2026-01-21 11:38:33.454775 | controller | ok 2026-01-21 11:38:33.502861 | 2026-01-21 11:38:33.503006 | TASK [cifmw_helpers : Fail if file is not yaml or yml extension] 2026-01-21 11:38:33.537710 | controller | skipping: Conditional result was False 2026-01-21 11:38:33.545346 | 2026-01-21 11:38:33.545442 | TASK [cifmw_helpers : Check if file is available] 2026-01-21 11:38:33.827181 | controller | ok 2026-01-21 11:38:33.842388 | 2026-01-21 11:38:33.842540 | TASK [cifmw_helpers : Create temporary directory] 2026-01-21 11:38:34.144376 | controller -> localhost | changed 2026-01-21 11:38:34.153057 | 2026-01-21 11:38:34.153158 | TASK [cifmw_helpers : Read vars] 2026-01-21 11:38:34.425429 | controller | Output suppressed because no_log was given 2026-01-21 11:38:34.439623 | 2026-01-21 11:38:34.439817 | TASK [cifmw_helpers : Create new variable files with content] 2026-01-21 11:38:34.943509 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:34.951200 | 2026-01-21 11:38:34.951286 | TASK [cifmw_helpers : Include vars] 2026-01-21 11:38:35.006280 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:35.017861 | 2026-01-21 11:38:35.018011 | TASK [cifmw_helpers : Remove temporary directory] 2026-01-21 11:38:35.274848 | controller | ok 2026-01-21 11:38:35.283561 | 2026-01-21 11:38:35.283652 | TASK [cifmw_helpers : Fail if file is not yaml or yml extension] 2026-01-21 11:38:35.308410 | controller | skipping: Conditional result was False 2026-01-21 11:38:35.319595 | 2026-01-21 11:38:35.319766 | TASK [cifmw_helpers : Check if file is available] 2026-01-21 11:38:35.615081 | controller | ok 2026-01-21 11:38:35.635330 | 2026-01-21 11:38:35.635617 | TASK [cifmw_helpers : Create temporary directory] 2026-01-21 11:38:35.920114 | controller -> localhost | changed 2026-01-21 11:38:35.940200 | 2026-01-21 11:38:35.940378 | TASK [cifmw_helpers : Read vars] 2026-01-21 11:38:36.203373 | controller | Output suppressed because no_log was given 2026-01-21 11:38:36.212415 | 2026-01-21 11:38:36.212503 | TASK [cifmw_helpers : Create new variable files with content] 2026-01-21 11:38:36.675207 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:36.693721 | 2026-01-21 11:38:36.693886 | TASK [cifmw_helpers : Include vars] 2026-01-21 11:38:36.732278 | controller -> localhost | Output suppressed because no_log was given 2026-01-21 11:38:36.748780 | 2026-01-21 11:38:36.748926 | TASK [cifmw_helpers : Remove temporary directory] 2026-01-21 11:38:37.031040 | controller | ok 2026-01-21 11:38:37.048199 | 2026-01-21 11:38:37.048296 | TASK [cifmw_setup : Set custom cifmw PATH reusable fact] 2026-01-21 11:38:37.083887 | controller | skipping: Conditional result was False 2026-01-21 11:38:37.093413 | 2026-01-21 11:38:37.093507 | TASK [cifmw_setup : Set destination folder for the logs] 2026-01-21 11:38:37.151370 | controller | ok 2026-01-21 11:38:37.160252 | 2026-01-21 11:38:37.160344 | TASK [artifacts : Verify if "success" flag exists after successful tests execution] 2026-01-21 11:38:37.416281 | controller | ok 2026-01-21 11:38:37.430733 | 2026-01-21 11:38:37.430877 | TASK [artifacts : Ensure we have at least minimal ansible facts] 2026-01-21 11:38:38.202732 | controller | ok 2026-01-21 11:38:38.218620 | 2026-01-21 11:38:38.218795 | LOOP [artifacts : Ensure base directory exists] 2026-01-21 11:38:38.516991 | controller | ok: "artifacts" 2026-01-21 11:38:38.756350 | controller | ok: "logs" 2026-01-21 11:38:38.779829 | 2026-01-21 11:38:38.780053 | TASK [artifacts : Load generated hook environment for further usage] 2026-01-21 11:38:38.839658 | controller | ERROR 2026-01-21 11:38:38.840030 | controller | { 2026-01-21 11:38:38.840092 | controller | "ansible_facts": {}, 2026-01-21 11:38:38.840132 | controller | "ansible_included_var_files": [], 2026-01-21 11:38:38.840168 | controller | "message": "/home/zuul/ci-framework-data/artifacts directory does not exist" 2026-01-21 11:38:38.840205 | controller | } 2026-01-21 11:38:38.840250 | controller | ERROR: Ignoring Errors 2026-01-21 11:38:38.853199 | 2026-01-21 11:38:38.853342 | TASK [artifacts : Gather parameter passed to the playbook] 2026-01-21 11:38:39.850231 | controller | changed 2026-01-21 11:38:39.863792 | 2026-01-21 11:38:39.863973 | TASK [artifacts : Gather facts] 2026-01-21 11:38:40.695375 | controller | changed 2026-01-21 11:38:40.712728 | 2026-01-21 11:38:40.712886 | TASK [artifacts : Copy some network configuration files] 2026-01-21 11:38:41.298165 | controller | changed 2026-01-21 11:38:41.314863 | 2026-01-21 11:38:41.315072 | TASK [artifacts : Get installed packages list] 2026-01-21 11:38:42.395320 | controller | ok 2026-01-21 11:38:42.453606 | 2026-01-21 11:38:42.453744 | TASK [artifacts : Output package list] 2026-01-21 11:38:43.339240 | controller | changed 2026-01-21 11:38:43.370883 | 2026-01-21 11:38:43.371065 | LOOP [artifacts : Cleanup unnecesary directories] 2026-01-21 11:38:43.741774 | controller | changed: "/home/zuul/ci-framework-data/artifacts/manifests/operator" 2026-01-21 11:38:43.742198 | controller | ok: All items complete 2026-01-21 11:38:43.742258 | 2026-01-21 11:38:43.804105 | 2026-01-21 11:38:43.804326 | TASK [os_must_gather : Ensure directories are present] 2026-01-21 11:38:44.117192 | controller | changed 2026-01-21 11:38:44.144452 | 2026-01-21 11:38:44.144569 | LOOP [os_must_gather : Construct project change list] 2026-01-21 11:38:44.296392 | controller | ok: 2026-01-21 11:38:44.296565 | controller | { 2026-01-21 11:38:44.296590 | controller | "branch": "main", 2026-01-21 11:38:44.296609 | controller | "change": "320", 2026-01-21 11:38:44.296626 | controller | "change_message": "Rabbitmq vhost and user support\n\nAdd new messagingBus and notificationsBus interfaces to hold cluster, user and vhost names for optional usage.\r\nThe controller adds these values to the TransportURL create request when present.\r\n\r\nAdditionally, we migrate RabbitMQ cluster name to RabbitMq config struct using DefaultRabbitMqConfig from infra-operator to automatically populate the new Cluster field from legacy RabbitMqClusterName.\r\n\r\nExample usage:\r\n\r\n```\r\n spec:\r\n messagingBus:\r\n cluster: rpc-rabbitmq\r\n user: rpc-user\r\n vhost: rpc-vhost\r\n notificationsBus:\r\n cluster: notifications-rabbitmq\r\n user: notifications-user\r\n vhost: notifications-vhost\r\n```\r\n\r\nJira: https://issues.redhat.com/browse/OSPRH-23882", 2026-01-21 11:38:44.296645 | controller | "change_url": "https://github.com/openstack-k8s-operators/watcher-operator/pull/320", 2026-01-21 11:38:44.296700 | controller | "commit_id": "111af6c57cd1be8032328ced6eb7a058a201c4b1", 2026-01-21 11:38:44.296724 | controller | "patchset": "111af6c57cd1be8032328ced6eb7a058a201c4b1", 2026-01-21 11:38:44.296741 | controller | "project": { 2026-01-21 11:38:44.296756 | controller | "canonical_hostname": "github.com", 2026-01-21 11:38:44.296772 | controller | "canonical_name": "github.com/openstack-k8s-operators/watcher-operator", 2026-01-21 11:38:44.296788 | controller | "name": "openstack-k8s-operators/watcher-operator", 2026-01-21 11:38:44.296803 | controller | "short_name": "watcher-operator", 2026-01-21 11:38:44.296819 | controller | "src_dir": "src/github.com/openstack-k8s-operators/watcher-operator" 2026-01-21 11:38:44.296834 | controller | }, 2026-01-21 11:38:44.296849 | controller | "topic": null 2026-01-21 11:38:44.296864 | controller | } 2026-01-21 11:38:44.358650 | 2026-01-21 11:38:44.358801 | TASK [os_must_gather : Get git tag for image tagging] 2026-01-21 11:38:45.000450 | controller | skipping: Conditional result was False 2026-01-21 11:38:45.020203 | 2026-01-21 11:38:45.020315 | TASK [os_must_gather : Debug git show-ref] 2026-01-21 11:38:45.045023 | controller | skipping: Conditional result was False 2026-01-21 11:38:45.068283 | 2026-01-21 11:38:45.068392 | TASK [os_must_gather : Build openstack-must-gather container] 2026-01-21 11:38:45.093003 | controller | skipping: Conditional result was False 2026-01-21 11:38:45.113138 | 2026-01-21 11:38:45.113245 | TASK [os_must_gather : Push openstack-must-gather container] 2026-01-21 11:38:45.137783 | controller | skipping: Conditional result was False 2026-01-21 11:38:45.157363 | 2026-01-21 11:38:45.157487 | TASK [os_must_gather : Set cifmw_os_must_gather_image to new image tag] 2026-01-21 11:38:45.192117 | controller | skipping: Conditional result was False 2026-01-21 11:38:45.219640 | 2026-01-21 11:38:45.219793 | TASK [os_must_gather : Check for oc command] 2026-01-21 11:38:45.839901 | controller | changed 2026-01-21 11:38:45.913713 | 2026-01-21 11:38:45.913856 | TASK [os_must_gather : Check if kubeconfig exists] 2026-01-21 11:38:46.306554 | controller | ok 2026-01-21 11:38:46.334641 | 2026-01-21 11:38:46.334811 | TASK [os_must_gather : Run openstack-must-gather command] 2026-01-21 12:23:47.023940 | controller | changed 2026-01-21 12:23:47.051150 | 2026-01-21 12:23:47.051247 | TASK [os_must_gather : Find existing os-must-gather directories] 2026-01-21 12:23:47.349114 | controller | ok: All paths examined 2026-01-21 12:23:47.366927 | 2026-01-21 12:23:47.366995 | TASK [os_must_gather : Create a symlink to newest os-must-gather directory] 2026-01-21 12:23:47.441555 | controller | ERROR 2026-01-21 12:23:47.441752 | controller | { 2026-01-21 12:23:47.441783 | controller | "msg": "The task includes an option with an undefined variable. The error was: Unable to look up a name or access an attribute in template string ({{ (_os_gather_latest_dir.files | sort(attribute='mtime', reverse=True) | first).path | basename }}).\nMake sure your variable name does not contain invalid characters like '-': expected str, bytes or os.PathLike object, not AnsibleUndefined. expected str, bytes or os.PathLike object, not AnsibleUndefined. Unable to look up a name or access an attribute in template string ({{ (_os_gather_latest_dir.files | sort(attribute='mtime', reverse=True) | first).path | basename }}).\nMake sure your variable name does not contain invalid characters like '-': expected str, bytes or os.PathLike object, not AnsibleUndefined. expected str, bytes or os.PathLike object, not AnsibleUndefined\n\nThe error appears to be in '/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/untrusted/project_0/github.com/openstack-k8s-operators/ci-framework/roles/os_must_gather/tasks/main.yml': line 96, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: Create a symlink to newest os-must-gather directory\n ^ here\n" 2026-01-21 12:23:47.441820 | controller | } 2026-01-21 12:23:47.441844 | controller | ERROR: Ignoring Errors 2026-01-21 12:23:47.475320 | 2026-01-21 12:23:47.475395 | TASK [artifacts : Create crc logs directory] 2026-01-21 12:23:47.825855 | controller | changed 2026-01-21 12:23:47.843266 | 2026-01-21 12:23:47.843331 | TASK [artifacts : Ensure controller knows CRC ssh keys] 2026-01-21 12:23:48.464124 | controller | changed 2026-01-21 12:23:48.503305 | 2026-01-21 12:23:48.503446 | TASK [artifacts : Recognize new keypair] 2026-01-21 12:23:48.774955 | controller | ok 2026-01-21 12:23:48.792149 | 2026-01-21 12:23:48.792218 | TASK [artifacts : Set fact if new keypair exists] 2026-01-21 12:23:48.837164 | controller | skipping: Conditional result was False 2026-01-21 12:23:48.857470 | 2026-01-21 12:23:48.857543 | TASK [artifacts : Prepare root ssh accesses] 2026-01-21 12:23:55.440330 | controller | changed 2026-01-21 12:23:55.477292 | 2026-01-21 12:23:55.477414 | TASK [artifacts : Copy logs from CRC VM] 2026-01-21 12:23:59.231322 | controller | changed 2026-01-21 12:23:59.264810 | 2026-01-21 12:23:59.264957 | TASK [artifacts : List all of the existing virtual machines] 2026-01-21 12:23:59.300835 | controller | skipping: Conditional result was False 2026-01-21 12:23:59.321265 | 2026-01-21 12:23:59.321343 | TASK [artifacts : Filter out edpm vm] 2026-01-21 12:23:59.356298 | controller | skipping: Conditional result was False 2026-01-21 12:23:59.377412 | 2026-01-21 12:23:59.377502 | TASK [artifacts : Slurp zuul inventory file] 2026-01-21 12:23:59.686389 | controller | ok 2026-01-21 12:23:59.704857 | 2026-01-21 12:23:59.704929 | LOOP [artifacts : Extract Compute and Networker from zuul mapping if any] 2026-01-21 12:23:59.782812 | 2026-01-21 12:23:59.782974 | LOOP [artifacts : Generate logs on edpm vm {{ item.split('/')[0] }}] 2026-01-21 12:23:59.870232 | 2026-01-21 12:23:59.870488 | LOOP [artifacts : Copy logs to host machine from {{ item.split('/')[0] }}] 2026-01-21 12:23:59.942173 | 2026-01-21 12:23:59.942413 | TASK [artifacts : Generate list of ansible logs to collect in home directory] 2026-01-21 12:24:00.298315 | controller | ok: All paths examined 2026-01-21 12:24:00.316270 | 2026-01-21 12:24:00.316342 | LOOP [artifacts : Copy ansible logs to logs directory] 2026-01-21 12:24:03.436227 | controller | changed: 2026-01-21 12:24:03.436500 | controller | { 2026-01-21 12:24:03.436555 | controller | "atime": 1768993595.8267531, 2026-01-21 12:24:03.436596 | controller | "ctime": 1768995504.4785476, 2026-01-21 12:24:03.436633 | controller | "dev": 64513, 2026-01-21 12:24:03.436671 | controller | "gid": 1000, 2026-01-21 12:24:03.436751 | controller | "gr_name": "zuul", 2026-01-21 12:24:03.436789 | controller | "inode": 4536298, 2026-01-21 12:24:03.436824 | controller | "isblk": false, 2026-01-21 12:24:03.436864 | controller | "ischr": false, 2026-01-21 12:24:03.436914 | controller | "isdir": false, 2026-01-21 12:24:03.436962 | controller | "isfifo": false, 2026-01-21 12:24:03.437011 | controller | "isgid": false, 2026-01-21 12:24:03.437046 | controller | "islnk": false, 2026-01-21 12:24:03.437079 | controller | "isreg": true, 2026-01-21 12:24:03.437112 | controller | "issock": false, 2026-01-21 12:24:03.437143 | controller | "isuid": false, 2026-01-21 12:24:03.437175 | controller | "mode": "0644", 2026-01-21 12:24:03.437207 | controller | "mtime": 1768995504.4785476, 2026-01-21 12:24:03.437240 | controller | "nlink": 1, 2026-01-21 12:24:03.437274 | controller | "path": "/home/zuul/ansible.log", 2026-01-21 12:24:03.437307 | controller | "pw_name": "zuul", 2026-01-21 12:24:03.437341 | controller | "rgrp": true, 2026-01-21 12:24:03.437374 | controller | "roth": true, 2026-01-21 12:24:03.437407 | controller | "rusr": true, 2026-01-21 12:24:03.437439 | controller | "size": 1858173, 2026-01-21 12:24:03.437471 | controller | "uid": 1000, 2026-01-21 12:24:03.437503 | controller | "wgrp": false, 2026-01-21 12:24:03.437536 | controller | "woth": false, 2026-01-21 12:24:03.437590 | controller | "wusr": true, 2026-01-21 12:24:03.437654 | controller | "xgrp": false, 2026-01-21 12:24:03.437728 | controller | "xoth": false, 2026-01-21 12:24:03.437767 | controller | "xusr": false 2026-01-21 12:24:03.437800 | controller | } 2026-01-21 12:24:03.437851 | 2026-01-21 12:24:03.488076 | 2026-01-21 12:24:03.488291 | TASK [artifacts : Ensure we have proper rights on the gathered content] 2026-01-21 12:24:05.626426 | controller | changed 2026-01-21 12:24:05.666503 | 2026-01-21 12:24:05.666715 | LOOP [artifacts : Mask secrets in yaml log files] 2026-01-21 12:24:40.668422 | controller | changed: "/home/zuul/ci-framework-data/logs" 2026-01-21 12:24:41.557927 | controller | changed: "/home/zuul/ci-framework-data/artifacts" 2026-01-21 12:24:41.617850 | 2026-01-21 12:24:41.617976 | LOOP [env_op_images : Ensure directory is present] 2026-01-21 12:24:41.917991 | controller | ok: "artifacts" 2026-01-21 12:24:41.918227 | controller | changed: All items complete 2026-01-21 12:24:41.918253 | 2026-01-21 12:24:42.141987 | controller | ok: "logs" 2026-01-21 12:24:42.213155 | 2026-01-21 12:24:42.213303 | TASK [env_op_images : Check if OpenStackControlPlane is setup] 2026-01-21 12:24:43.383959 | controller | changed 2026-01-21 12:24:43.433777 | 2026-01-21 12:24:43.433900 | TASK [env_op_images : Get images from the CSV] 2026-01-21 12:24:44.097850 | controller | skipping: Conditional result was False 2026-01-21 12:24:44.144386 | 2026-01-21 12:24:44.144590 | TASK [env_op_images : Get the images name] 2026-01-21 12:24:44.794620 | controller | skipping: Conditional result was False 2026-01-21 12:24:44.821628 | 2026-01-21 12:24:44.821770 | TASK [env_op_images : Extract env variable name and images] 2026-01-21 12:24:44.847232 | controller | skipping: Conditional result was False 2026-01-21 12:24:44.875269 | 2026-01-21 12:24:44.875382 | TASK [env_op_images : Get all pods from all namespaces to find openstack-operator-index] 2026-01-21 12:24:44.911519 | controller | skipping: Conditional result was False 2026-01-21 12:24:44.941119 | 2026-01-21 12:24:44.941238 | TASK [env_op_images : Retrieve openstack-operator-index pod] 2026-01-21 12:24:44.966633 | controller | skipping: Conditional result was False 2026-01-21 12:24:44.986909 | 2026-01-21 12:24:44.986983 | TASK [env_op_images : Get all the pods in openstack-operator namespace] 2026-01-21 12:24:45.022660 | controller | skipping: Conditional result was False 2026-01-21 12:24:45.042630 | 2026-01-21 12:24:45.042724 | TASK [env_op_images : Get operator images and pods] 2026-01-21 12:24:45.067376 | controller | skipping: Conditional result was False 2026-01-21 12:24:45.090575 | 2026-01-21 12:24:45.090687 | LOOP [env_op_images : Add operator images to the dictionary] 2026-01-21 12:24:45.182972 | 2026-01-21 12:24:45.183140 | TASK [env_op_images : Write images to file] 2026-01-21 12:24:45.208502 | controller | skipping: Conditional result was False 2026-01-21 12:24:45.246053 | 2026-01-21 12:24:45.246150 | TASK [cifmw_setup : Create a versioned log folder] 2026-01-21 12:24:45.528209 | controller | changed 2026-01-21 12:24:45.565000 | 2026-01-21 12:24:45.565209 | TASK [cifmw_setup : Return a list of log files in home directory] 2026-01-21 12:24:45.847055 | controller | ok: All paths examined 2026-01-21 12:24:45.874507 | 2026-01-21 12:24:45.874663 | TASK [cifmw_setup : Ensure ansible facts cache exists] 2026-01-21 12:24:46.181596 | controller | ok 2026-01-21 12:24:46.209696 | 2026-01-21 12:24:46.209859 | LOOP [cifmw_setup : Copy logs to proper location] 2026-01-21 12:24:46.708384 | controller | changed: 2026-01-21 12:24:46.708656 | controller | { 2026-01-21 12:24:46.708716 | controller | "atime": 1768998243.293176, 2026-01-21 12:24:46.708744 | controller | "ctime": 1768995504.4785476, 2026-01-21 12:24:46.708768 | controller | "dev": 64513, 2026-01-21 12:24:46.708791 | controller | "gid": 1000, 2026-01-21 12:24:46.708815 | controller | "gr_name": "zuul", 2026-01-21 12:24:46.708837 | controller | "inode": 4536298, 2026-01-21 12:24:46.708868 | controller | "isblk": false, 2026-01-21 12:24:46.708890 | controller | "ischr": false, 2026-01-21 12:24:46.708913 | controller | "isdir": false, 2026-01-21 12:24:46.708934 | controller | "isfifo": false, 2026-01-21 12:24:46.708955 | controller | "isgid": false, 2026-01-21 12:24:46.708976 | controller | "islnk": false, 2026-01-21 12:24:46.708997 | controller | "isreg": true, 2026-01-21 12:24:46.709018 | controller | "issock": false, 2026-01-21 12:24:46.709039 | controller | "isuid": false, 2026-01-21 12:24:46.709060 | controller | "mode": "0644", 2026-01-21 12:24:46.709082 | controller | "mtime": 1768995504.4785476, 2026-01-21 12:24:46.709103 | controller | "nlink": 1, 2026-01-21 12:24:46.709129 | controller | "path": "/home/zuul/ansible.log", 2026-01-21 12:24:46.709153 | controller | "pw_name": "zuul", 2026-01-21 12:24:46.709176 | controller | "rgrp": true, 2026-01-21 12:24:46.709198 | controller | "roth": true, 2026-01-21 12:24:46.709219 | controller | "rusr": true, 2026-01-21 12:24:46.709241 | controller | "size": 1858173, 2026-01-21 12:24:46.709262 | controller | "uid": 1000, 2026-01-21 12:24:46.709284 | controller | "wgrp": false, 2026-01-21 12:24:46.709305 | controller | "woth": false, 2026-01-21 12:24:46.709326 | controller | "wusr": true, 2026-01-21 12:24:46.709348 | controller | "xgrp": false, 2026-01-21 12:24:46.709370 | controller | "xoth": false, 2026-01-21 12:24:46.709391 | controller | "xusr": false 2026-01-21 12:24:46.709412 | controller | } 2026-01-21 12:24:46.709442 | 2026-01-21 12:24:46.739238 | 2026-01-21 12:24:46.739379 | LOOP [cifmw_setup : Remove original log from home directory] 2026-01-21 12:24:46.998671 | controller | changed: 2026-01-21 12:24:46.998979 | controller | { 2026-01-21 12:24:46.999020 | controller | "atime": 1768998243.293176, 2026-01-21 12:24:46.999046 | controller | "ctime": 1768995504.4785476, 2026-01-21 12:24:46.999069 | controller | "dev": 64513, 2026-01-21 12:24:46.999092 | controller | "gid": 1000, 2026-01-21 12:24:46.999114 | controller | "gr_name": "zuul", 2026-01-21 12:24:46.999135 | controller | "inode": 4536298, 2026-01-21 12:24:46.999155 | controller | "isblk": false, 2026-01-21 12:24:46.999175 | controller | "ischr": false, 2026-01-21 12:24:46.999195 | controller | "isdir": false, 2026-01-21 12:24:46.999214 | controller | "isfifo": false, 2026-01-21 12:24:46.999235 | controller | "isgid": false, 2026-01-21 12:24:46.999256 | controller | "islnk": false, 2026-01-21 12:24:46.999276 | controller | "isreg": true, 2026-01-21 12:24:46.999296 | controller | "issock": false, 2026-01-21 12:24:46.999316 | controller | "isuid": false, 2026-01-21 12:24:46.999336 | controller | "mode": "0644", 2026-01-21 12:24:46.999356 | controller | "mtime": 1768995504.4785476, 2026-01-21 12:24:46.999376 | controller | "nlink": 1, 2026-01-21 12:24:46.999396 | controller | "path": "/home/zuul/ansible.log", 2026-01-21 12:24:46.999416 | controller | "pw_name": "zuul", 2026-01-21 12:24:46.999435 | controller | "rgrp": true, 2026-01-21 12:24:46.999455 | controller | "roth": true, 2026-01-21 12:24:46.999475 | controller | "rusr": true, 2026-01-21 12:24:46.999495 | controller | "size": 1858173, 2026-01-21 12:24:46.999515 | controller | "uid": 1000, 2026-01-21 12:24:46.999534 | controller | "wgrp": false, 2026-01-21 12:24:46.999554 | controller | "woth": false, 2026-01-21 12:24:46.999573 | controller | "wusr": true, 2026-01-21 12:24:46.999593 | controller | "xgrp": false, 2026-01-21 12:24:46.999612 | controller | "xoth": false, 2026-01-21 12:24:46.999632 | controller | "xusr": false 2026-01-21 12:24:46.999651 | controller | } 2026-01-21 12:24:47.028163 | 2026-01-21 12:24:47.028270 | TASK [cifmw_setup : Copy facts to dated directory] 2026-01-21 12:24:47.477794 | controller | changed 2026-01-21 12:24:47.561486 | 2026-01-21 12:24:47.561750 | TASK [run_hook : Assert parameters are valid] 2026-01-21 12:24:47.710827 | controller | ok: All assertions passed 2026-01-21 12:24:47.737558 | 2026-01-21 12:24:47.737709 | TASK [run_hook : Assert single hooks are all mappings] 2026-01-21 12:24:47.876119 | controller | ok: All assertions passed 2026-01-21 12:24:47.907127 | 2026-01-21 12:24:47.907633 | LOOP [run_hook : Loop on hooks for post_logs] 2026-01-21 12:24:48.119016 | 2026-01-21 12:24:48.119198 | PLAY [Run ci/playbooks/e2e-collect-logs.yml on CRC host] 2026-01-21 12:24:48.144121 | 2026-01-21 12:24:48.144203 | TASK [Get kubelet journalctl logs] 2026-01-21 12:24:49.686789 | crc | Output suppressed because no_log was given 2026-01-21 12:24:49.700926 | 2026-01-21 12:24:49.701062 | TASK [Compress logs bigger than 2MB] 2026-01-21 12:24:50.808942 | crc | changed 2026-01-21 12:24:50.825344 | 2026-01-21 12:24:50.825478 | TASK [Copy files from workspace on node] 2026-01-21 12:24:50.868889 | crc | ok 2026-01-21 12:24:50.914226 | 2026-01-21 12:24:50.914327 | TASK [fetch-output : Set log path for multiple nodes] 2026-01-21 12:24:50.975969 | crc | ok 2026-01-21 12:24:50.992386 | 2026-01-21 12:24:50.992524 | TASK [fetch-output : Set log path for single node] 2026-01-21 12:24:51.028721 | crc | skipping: Conditional result was False 2026-01-21 12:24:51.039383 | 2026-01-21 12:24:51.039482 | LOOP [fetch-output : Ensure local output dirs] 2026-01-21 12:24:51.344080 | crc -> localhost | changed: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/logs/crc" 2026-01-21 12:24:51.344335 | 2026-01-21 12:24:51.552441 | crc -> localhost | changed: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/artifacts" 2026-01-21 12:24:51.757084 | crc -> localhost | changed: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/docs" 2026-01-21 12:24:51.776830 | 2026-01-21 12:24:51.776948 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-01-21 12:24:52.560235 | crc | changed: 2026-01-21 12:24:52.560713 | crc | .d..t...... ./ 2026-01-21 12:24:52.560794 | crc | >f+++++++++ kubelet.log.gz 2026-01-21 12:24:53.158074 | crc | changed: .d..t...... ./ 2026-01-21 12:24:53.711765 | crc | changed: .d..t...... ./ 2026-01-21 12:24:53.747554 | 2026-01-21 12:24:53.747624 | PLAY RECAP 2026-01-21 12:24:53.747664 | controller | ok: 74 changed: 28 unreachable: 0 failed: 0 skipped: 33 rescued: 0 ignored: 2 2026-01-21 12:24:53.747710 | crc | ok: 5 changed: 4 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-01-21 12:24:53.747729 | 2026-01-21 12:24:53.916207 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/e2e-collect-logs.yml@main] 2026-01-21 12:24:53.924930 | POST-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/collect-logs.yml@main] 2026-01-21 12:24:54.567791 | 2026-01-21 12:24:54.567899 | PLAY [Run ci/playbooks/collect-logs.yml] 2026-01-21 12:24:54.662478 | 2026-01-21 12:24:54.662635 | TASK [Filter out host if needed] 2026-01-21 12:24:54.717375 | controller | skipping: end_host conditional evaluated to False, continuing execution for controller 2026-01-21 12:24:54.723888 | 2026-01-21 12:24:54.723958 | TASK [Filter out host if needed] 2026-01-21 12:24:54.756153 | 2026-01-21 12:24:54.756257 | TASK [Ensure file is present] 2026-01-21 12:24:55.289729 | controller | ok 2026-01-21 12:24:55.314364 | 2026-01-21 12:24:55.314447 | TASK [Manage molecule report file] 2026-01-21 12:24:56.009921 | controller | skipping: Conditional result was False 2026-01-21 12:24:56.034689 | 2026-01-21 12:24:56.034791 | TASK [Check if we get ci-framework-data basedir] 2026-01-21 12:24:56.328239 | controller | ok 2026-01-21 12:24:56.351797 | 2026-01-21 12:24:56.351873 | TASK [Create ci-framework-data log directory for zuul] 2026-01-21 12:24:56.944482 | controller | changed 2026-01-21 12:24:56.969043 | 2026-01-21 12:24:56.969128 | TASK [Copy ci-framework interesting files] 2026-01-21 12:24:57.619035 | controller | cp: cannot stat '/home/zuul/ci-framework-data/tests': No such file or directory 2026-01-21 12:25:07.705472 | controller | changed 2026-01-21 12:25:07.755642 | 2026-01-21 12:25:07.755818 | TASK [Get SELinux listing] 2026-01-21 12:25:18.502860 | controller | changed 2026-01-21 12:25:18.560426 | 2026-01-21 12:25:18.560557 | TASK [Generate log index] 2026-01-21 12:25:19.561726 | controller | changed 2026-01-21 12:25:19.591870 | 2026-01-21 12:25:19.591965 | TASK [Get some env related data] 2026-01-21 12:25:20.566507 | controller | /home/zuul/.local/bin/ansible 2026-01-21 12:25:31.819278 | controller | changed 2026-01-21 12:25:31.843127 | 2026-01-21 12:25:31.843215 | TASK [Generate list of logs to collect in home directory] 2026-01-21 12:25:32.266468 | controller | ok: All paths examined 2026-01-21 12:25:32.301171 | 2026-01-21 12:25:32.301291 | LOOP [Copy logs from home directory] 2026-01-21 12:25:32.351901 | 2026-01-21 12:25:32.352057 | TASK [Copy crio stats log file] 2026-01-21 12:25:32.377258 | controller | skipping: Conditional result was False 2026-01-21 12:25:32.415524 | 2026-01-21 12:25:32.415693 | TASK [Get SELinux related data] 2026-01-21 12:25:32.924296 | controller | 2026-01-21 12:25:43.369563 | controller | ERROR 2026-01-21 12:25:43.369783 | controller | { 2026-01-21 12:25:43.369817 | controller | "delta": "0:00:00.013877", 2026-01-21 12:25:43.369836 | controller | "end": "2026-01-21 12:25:32.926209", 2026-01-21 12:25:43.369854 | controller | "msg": "non-zero return code", 2026-01-21 12:25:43.369870 | controller | "rc": 1, 2026-01-21 12:25:43.369887 | controller | "start": "2026-01-21 12:25:32.912332" 2026-01-21 12:25:43.369902 | controller | } 2026-01-21 12:25:43.369928 | controller | ERROR: Ignoring Errors 2026-01-21 12:25:43.393458 | 2026-01-21 12:25:43.393529 | TASK [Create system configuration directory] 2026-01-21 12:25:43.740076 | controller | changed 2026-01-21 12:25:43.764089 | 2026-01-21 12:25:43.764170 | TASK [Get some of the system configurations] 2026-01-21 12:25:54.564073 | controller | changed 2026-01-21 12:25:54.592574 | 2026-01-21 12:25:54.592761 | TASK [Copy generated documentation if available] 2026-01-21 12:25:54.634937 | controller | skipping: Conditional result was False 2026-01-21 12:25:54.678141 | 2026-01-21 12:25:54.678272 | TASK [Copy generated AsciiDoc documentation if available] 2026-01-21 12:25:54.694948 | controller | skipping: Conditional result was False 2026-01-21 12:25:54.742429 | 2026-01-21 12:25:54.742573 | TASK [Compress logs bigger than 2MB] 2026-01-21 12:26:08.611519 | controller | changed 2026-01-21 12:26:08.663717 | 2026-01-21 12:26:08.663845 | TASK [Copy files from workspace on node] 2026-01-21 12:26:08.745087 | controller | ok 2026-01-21 12:26:08.806865 | 2026-01-21 12:26:08.807000 | TASK [fetch-output : Set log path for multiple nodes] 2026-01-21 12:26:08.866125 | controller | ok 2026-01-21 12:26:08.889033 | 2026-01-21 12:26:08.889121 | TASK [fetch-output : Set log path for single node] 2026-01-21 12:26:08.914085 | controller | skipping: Conditional result was False 2026-01-21 12:26:08.941198 | 2026-01-21 12:26:08.941430 | LOOP [fetch-output : Ensure local output dirs] 2026-01-21 12:26:09.199757 | controller -> localhost | changed: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/logs/controller" 2026-01-21 12:26:09.200022 | 2026-01-21 12:26:09.441267 | controller -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/artifacts" 2026-01-21 12:26:09.642037 | controller -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/docs" 2026-01-21 12:26:09.714730 | 2026-01-21 12:26:09.714855 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-01-21 12:26:11.264131 | controller | changed: 2026-01-21 12:26:11.264411 | controller | .d..t...... ./ 2026-01-21 12:26:11.264442 | controller | >f+++++++++ README.html 2026-01-21 12:26:11.264464 | controller | >f+++++++++ dmesg.log 2026-01-21 12:26:11.264485 | controller | >f+++++++++ installed-pkgs.log 2026-01-21 12:26:11.264504 | controller | >f+++++++++ python.log 2026-01-21 12:26:11.264521 | controller | >f+++++++++ registries.conf 2026-01-21 12:26:11.264539 | controller | >f+++++++++ selinux-denials.log 2026-01-21 12:26:11.264557 | controller | >f+++++++++ selinux-listing.log 2026-01-21 12:26:11.264583 | controller | cd+++++++++ ci-framework-data/ 2026-01-21 12:26:11.264601 | controller | cd+++++++++ ci-framework-data/artifacts/ 2026-01-21 12:26:11.264618 | controller | >f+++++++++ ci-framework-data/artifacts/ansible-facts.yml 2026-01-21 12:26:11.264634 | controller | >f+++++++++ ci-framework-data/artifacts/ansible-vars.yml 2026-01-21 12:26:11.264651 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_000_check_for_oc.sh 2026-01-21 12:26:11.264667 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_000_copy_logs_from_crc.sh 2026-01-21 12:26:11.264705 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_000_prepare_root_ssh.sh 2026-01-21 12:26:11.264724 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_000_run_download.sh 2026-01-21 12:26:11.264742 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_000_run_openstack_must_gather.sh 2026-01-21 12:26:11.264758 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_001_run_hook_without_retry.sh 2026-01-21 12:26:11.264775 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_002_fetch_openshift.sh 2026-01-21 12:26:11.264791 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_003_run_openstack.sh 2026-01-21 12:26:11.264808 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_004_run_hook_without_retry_deploy.sh 2026-01-21 12:26:11.264825 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_005_stop_watcher.sh 2026-01-21 12:26:11.264842 | controller | >f+++++++++ ci-framework-data/artifacts/ci_script_006_install_watcher_operator_in.sh 2026-01-21 12:26:11.264858 | controller | >f+++++++++ ci-framework-data/artifacts/hosts 2026-01-21 12:26:11.264875 | controller | >f+++++++++ ci-framework-data/artifacts/install_yamls.sh 2026-01-21 12:26:11.264892 | controller | >f+++++++++ ci-framework-data/artifacts/installed-packages.yml 2026-01-21 12:26:11.264908 | controller | >f+++++++++ ci-framework-data/artifacts/ip-network.txt 2026-01-21 12:26:11.264924 | controller | >f+++++++++ ci-framework-data/artifacts/resolv.conf 2026-01-21 12:26:11.264940 | controller | >f+++++++++ ci-framework-data/artifacts/zuul_inventory.yml 2026-01-21 12:26:11.264957 | controller | cd+++++++++ ci-framework-data/artifacts/NetworkManager/ 2026-01-21 12:26:11.264973 | controller | >f+++++++++ ci-framework-data/artifacts/NetworkManager/ci-private-network.nmconnection 2026-01-21 12:26:11.264990 | controller | >f+++++++++ ci-framework-data/artifacts/NetworkManager/ens3.nmconnection 2026-01-21 12:26:11.265006 | controller | cd+++++++++ ci-framework-data/artifacts/ansible_facts.2026-01-21_12-24/ 2026-01-21 12:26:11.265022 | controller | cd+++++++++ ci-framework-data/artifacts/ansible_facts.2026-01-21_12-24/ansible_facts_cache/ 2026-01-21 12:26:11.265039 | controller | >f+++++++++ ci-framework-data/artifacts/ansible_facts.2026-01-21_12-24/ansible_facts_cache/controller 2026-01-21 12:26:11.265056 | controller | >f+++++++++ ci-framework-data/artifacts/ansible_facts.2026-01-21_12-24/ansible_facts_cache/localhost 2026-01-21 12:26:11.265072 | controller | cd+++++++++ ci-framework-data/artifacts/ci-env/ 2026-01-21 12:26:11.265089 | controller | >f+++++++++ ci-framework-data/artifacts/ci-env/networking-info.yml 2026-01-21 12:26:11.265106 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/ 2026-01-21 12:26:11.265125 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/cert-manager-operator/ 2026-01-21 12:26:11.265143 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/cert-manager-operator/namespace.yaml 2026-01-21 12:26:11.265161 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/cert-manager-operator/cert-manager/ 2026-01-21 12:26:11.265181 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/cert-manager-operator/cert-manager/op/ 2026-01-21 12:26:11.265198 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/cert-manager-operator/cert-manager/op/operatorgroup.yaml 2026-01-21 12:26:11.265220 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/cert-manager-operator/cert-manager/op/subscription.yaml 2026-01-21 12:26:11.265237 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/metallb-system/ 2026-01-21 12:26:11.265255 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/namespace.yaml 2026-01-21 12:26:11.265272 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/ 2026-01-21 12:26:11.265288 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/ 2026-01-21 12:26:11.265305 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/bgpadvertisement.yaml 2026-01-21 12:26:11.265321 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/bgpextras.yaml 2026-01-21 12:26:11.265337 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/bgppeers.yaml 2026-01-21 12:26:11.265354 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/deploy_operator.yaml 2026-01-21 12:26:11.265370 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/ipaddresspools.yaml 2026-01-21 12:26:11.265386 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/metallb-system/metallb/cr/l2advertisement.yaml 2026-01-21 12:26:11.265403 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openshift-nmstate/ 2026-01-21 12:26:11.265420 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openshift-nmstate/namespace.yaml 2026-01-21 12:26:11.265437 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openshift-nmstate/nmstate/ 2026-01-21 12:26:11.265454 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openshift-nmstate/nmstate/cr/ 2026-01-21 12:26:11.265471 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openshift-nmstate/nmstate/cr/deploy_operator.yaml 2026-01-21 12:26:11.265488 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/ 2026-01-21 12:26:11.265504 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/namespace.yaml 2026-01-21 12:26:11.265521 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/metallb/ 2026-01-21 12:26:11.265537 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/metallb/op/ 2026-01-21 12:26:11.265553 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/metallb/op/operatorgroup.yaml 2026-01-21 12:26:11.265569 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/metallb/op/subscription.yaml 2026-01-21 12:26:11.265586 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/nmstate/ 2026-01-21 12:26:11.265602 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/nmstate/op/ 2026-01-21 12:26:11.265618 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/nmstate/op/operatorgroup.yaml 2026-01-21 12:26:11.265635 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/nmstate/op/subscription.yaml 2026-01-21 12:26:11.265651 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/openstack/ 2026-01-21 12:26:11.265667 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/openstack/op/ 2026-01-21 12:26:11.265701 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/openstack/op/catalogsource.yaml 2026-01-21 12:26:11.265719 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/openstack/op/operatorgroup.yaml 2026-01-21 12:26:11.265736 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack-operators/openstack/op/subscription.yaml 2026-01-21 12:26:11.265753 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/ 2026-01-21 12:26:11.265773 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/namespace.yaml 2026-01-21 12:26:11.265791 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/cr/ 2026-01-21 12:26:11.265808 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/ 2026-01-21 12:26:11.265825 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/ 2026-01-21 12:26:11.265842 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/ctlplane.yaml 2026-01-21 12:26:11.265859 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/datacentre.yaml 2026-01-21 12:26:11.265875 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/designate.yaml 2026-01-21 12:26:11.265891 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/designateext.yaml 2026-01-21 12:26:11.265908 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/internalapi.yaml 2026-01-21 12:26:11.265924 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/octavia.yaml 2026-01-21 12:26:11.265941 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/storage.yaml 2026-01-21 12:26:11.265957 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/storagemgmt.yaml 2026-01-21 12:26:11.265973 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/netattach/cr/tenant.yaml 2026-01-21 12:26:11.265989 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/nncp/ 2026-01-21 12:26:11.266006 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/nncp/cr/ 2026-01-21 12:26:11.266022 | controller | >f+++++++++ ci-framework-data/artifacts/manifests/openstack/nncp/cr/crc_nncp.yaml 2026-01-21 12:26:11.266039 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/openstack/ 2026-01-21 12:26:11.266055 | controller | cd+++++++++ ci-framework-data/artifacts/manifests/openstack/openstack/cr/ 2026-01-21 12:26:11.266071 | controller | cd+++++++++ ci-framework-data/artifacts/parameters/ 2026-01-21 12:26:11.266087 | controller | >f+++++++++ ci-framework-data/artifacts/parameters/custom-params.yml 2026-01-21 12:26:11.266103 | controller | >f+++++++++ ci-framework-data/artifacts/parameters/install-yamls-params.yml 2026-01-21 12:26:11.266120 | controller | >f+++++++++ ci-framework-data/artifacts/parameters/openshift-login-params.yml 2026-01-21 12:26:11.266135 | controller | >f+++++++++ ci-framework-data/artifacts/parameters/zuul-params.yml 2026-01-21 12:26:11.266151 | controller | cd+++++++++ ci-framework-data/artifacts/repositories/ 2026-01-21 12:26:11.266167 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/delorean-antelope-testing.repo 2026-01-21 12:26:11.266183 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/delorean.repo 2026-01-21 12:26:11.266199 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/delorean.repo.md5 2026-01-21 12:26:11.266215 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/repo-setup-centos-appstream.repo 2026-01-21 12:26:11.266231 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/repo-setup-centos-baseos.repo 2026-01-21 12:26:11.266247 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/repo-setup-centos-highavailability.repo 2026-01-21 12:26:11.266263 | controller | >f+++++++++ ci-framework-data/artifacts/repositories/repo-setup-centos-powertools.repo 2026-01-21 12:26:11.266280 | controller | cd+++++++++ ci-framework-data/artifacts/roles/ 2026-01-21 12:26:11.266296 | controller | cd+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/ 2026-01-21 12:26:11.266313 | controller | cd+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/ 2026-01-21 12:26:11.266331 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_all.yml 2026-01-21 12:26:11.266354 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee.yml 2026-01-21 12:26:11.266372 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee_cleanup.yml 2026-01-21 12:26:11.266389 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee_kuttl.yml 2026-01-21 12:26:11.266406 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee_kuttl_cleanup.yml 2026-01-21 12:26:11.266422 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee_kuttl_prep.yml 2026-01-21 12:26:11.266439 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee_kuttl_run.yml 2026-01-21 12:26:11.266456 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ansibleee_prep.yml 2026-01-21 12:26:11.266472 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_attach_default_interface.yml 2026-01-21 12:26:11.266489 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_attach_default_interface_cleanup.yml 2026-01-21 12:26:11.266505 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican.yml 2026-01-21 12:26:11.266522 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_cleanup.yml 2026-01-21 12:26:11.266539 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_deploy.yml 2026-01-21 12:26:11.266556 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_deploy_cleanup.yml 2026-01-21 12:26:11.266573 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_deploy_prep.yml 2026-01-21 12:26:11.266590 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_deploy_validate.yml 2026-01-21 12:26:11.266607 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_kuttl.yml 2026-01-21 12:26:11.266623 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_kuttl_run.yml 2026-01-21 12:26:11.266641 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_barbican_prep.yml 2026-01-21 12:26:11.266657 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_baremetal.yml 2026-01-21 12:26:11.266674 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_baremetal_cleanup.yml 2026-01-21 12:26:11.266710 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_baremetal_prep.yml 2026-01-21 12:26:11.266728 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas.yml 2026-01-21 12:26:11.266745 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_baremetal_net_nad.yml 2026-01-21 12:26:11.266762 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_baremetal_net_nad_cleanup.yml 2026-01-21 12:26:11.266778 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_cleanup.yml 2026-01-21 12:26:11.266794 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_crc_attach_network.yml 2026-01-21 12:26:11.266811 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_crc_attach_network_cleanup.yml 2026-01-21 12:26:11.266829 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_crc_baremetal_bridge.yml 2026-01-21 12:26:11.266848 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_crc_baremetal_bridge_cleanup.yml 2026-01-21 12:26:11.266868 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_generate_nodes_yaml.yml 2026-01-21 12:26:11.266884 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_metallb.yml 2026-01-21 12:26:11.266900 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_metallb_cleanup.yml 2026-01-21 12:26:11.266916 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_network.yml 2026-01-21 12:26:11.266931 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_network_cleanup.yml 2026-01-21 12:26:11.266947 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_route_crc_and_crc_bmaas_networks.yml 2026-01-21 12:26:11.266963 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_route_crc_and_crc_bmaas_networks_cleanup.yml 2026-01-21 12:26:11.266979 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_sushy_emulator.yml 2026-01-21 12:26:11.266995 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_sushy_emulator_cleanup.yml 2026-01-21 12:26:11.267010 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_sushy_emulator_wait.yml 2026-01-21 12:26:11.267026 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_virtual_bms.yml 2026-01-21 12:26:11.267042 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_bmaas_virtual_bms_cleanup.yml 2026-01-21 12:26:11.267058 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ceph.yml 2026-01-21 12:26:11.267075 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ceph_cleanup.yml 2026-01-21 12:26:11.267091 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ceph_help.yml 2026-01-21 12:26:11.267107 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_certmanager.yml 2026-01-21 12:26:11.267124 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_certmanager_cleanup.yml 2026-01-21 12:26:11.267140 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cifmw_cleanup.yml 2026-01-21 12:26:11.267156 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cifmw_prepare.yml 2026-01-21 12:26:11.267172 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder.yml 2026-01-21 12:26:11.267187 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_cleanup.yml 2026-01-21 12:26:11.267202 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_deploy.yml 2026-01-21 12:26:11.267217 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_deploy_cleanup.yml 2026-01-21 12:26:11.267232 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_deploy_prep.yml 2026-01-21 12:26:11.267247 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_kuttl.yml 2026-01-21 12:26:11.267263 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_kuttl_run.yml 2026-01-21 12:26:11.267279 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cinder_prep.yml 2026-01-21 12:26:11.267295 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_cleanup.yml 2026-01-21 12:26:11.267315 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc.yml 2026-01-21 12:26:11.267334 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_attach_default_interface.yml 2026-01-21 12:26:11.267351 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_attach_default_interface_cleanup.yml 2026-01-21 12:26:11.267368 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_bmo_cleanup.yml 2026-01-21 12:26:11.267385 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_bmo_setup.yml 2026-01-21 12:26:11.267401 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_cleanup.yml 2026-01-21 12:26:11.267417 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_scrub.yml 2026-01-21 12:26:11.267434 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_storage.yml 2026-01-21 12:26:11.267451 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_storage_cleanup.yml 2026-01-21 12:26:11.267468 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_storage_cleanup_with_retries.yml 2026-01-21 12:26:11.267484 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_storage_release.yml 2026-01-21 12:26:11.267501 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_crc_storage_with_retries.yml 2026-01-21 12:26:11.267518 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_deploy_cleanup.yml 2026-01-21 12:26:11.267534 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate.yml 2026-01-21 12:26:11.267551 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_cleanup.yml 2026-01-21 12:26:11.267567 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_deploy.yml 2026-01-21 12:26:11.267583 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_deploy_cleanup.yml 2026-01-21 12:26:11.267600 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_deploy_prep.yml 2026-01-21 12:26:11.267616 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_kuttl.yml 2026-01-21 12:26:11.267632 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_kuttl_run.yml 2026-01-21 12:26:11.267648 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_designate_prep.yml 2026-01-21 12:26:11.267665 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_dns_deploy.yml 2026-01-21 12:26:11.267704 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_dns_deploy_cleanup.yml 2026-01-21 12:26:11.267723 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_dns_deploy_prep.yml 2026-01-21 12:26:11.267741 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_download_tools.yml 2026-01-21 12:26:11.267757 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_ansible_runner.yml 2026-01-21 12:26:11.267774 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_baremetal_compute.yml 2026-01-21 12:26:11.267791 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_compute.yml 2026-01-21 12:26:11.267808 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_compute_bootc.yml 2026-01-21 12:26:11.267828 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_compute_cleanup.yml 2026-01-21 12:26:11.267846 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_compute_repos.yml 2026-01-21 12:26:11.267876 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_computes_bgp.yml 2026-01-21 12:26:11.267894 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy.yml 2026-01-21 12:26:11.267911 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_baremetal.yml 2026-01-21 12:26:11.267928 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_baremetal_prep.yml 2026-01-21 12:26:11.267945 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_cleanup.yml 2026-01-21 12:26:11.267961 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_generate_keys.yml 2026-01-21 12:26:11.267977 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_instance.yml 2026-01-21 12:26:11.267994 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_networker.yml 2026-01-21 12:26:11.268010 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_networker_cleanup.yml 2026-01-21 12:26:11.268027 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_networker_prep.yml 2026-01-21 12:26:11.268065 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_deploy_prep.yml 2026-01-21 12:26:11.268081 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_networker.yml 2026-01-21 12:26:11.268097 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_networker_cleanup.yml 2026-01-21 12:26:11.268114 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_nova_discover_hosts.yml 2026-01-21 12:26:11.268130 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_patch_ansible_runner_image.yml 2026-01-21 12:26:11.268146 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_register_dns.yml 2026-01-21 12:26:11.268161 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_wait_deploy.yml 2026-01-21 12:26:11.268177 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_edpm_wait_deploy_baremetal.yml 2026-01-21 12:26:11.268193 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance.yml 2026-01-21 12:26:11.268209 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_cleanup.yml 2026-01-21 12:26:11.268225 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_deploy.yml 2026-01-21 12:26:11.268241 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_deploy_cleanup.yml 2026-01-21 12:26:11.268256 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_deploy_prep.yml 2026-01-21 12:26:11.268272 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_kuttl.yml 2026-01-21 12:26:11.268288 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_kuttl_run.yml 2026-01-21 12:26:11.268304 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_glance_prep.yml 2026-01-21 12:26:11.268319 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat.yml 2026-01-21 12:26:11.268343 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_cleanup.yml 2026-01-21 12:26:11.268359 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_deploy.yml 2026-01-21 12:26:11.268375 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_deploy_cleanup.yml 2026-01-21 12:26:11.268390 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_deploy_prep.yml 2026-01-21 12:26:11.268406 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_kuttl.yml 2026-01-21 12:26:11.268421 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_kuttl_crc.yml 2026-01-21 12:26:11.268437 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_kuttl_run.yml 2026-01-21 12:26:11.268452 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_heat_prep.yml 2026-01-21 12:26:11.268467 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_help.yml 2026-01-21 12:26:11.268483 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon.yml 2026-01-21 12:26:11.268499 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_cleanup.yml 2026-01-21 12:26:11.268515 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_deploy.yml 2026-01-21 12:26:11.268531 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_deploy_cleanup.yml 2026-01-21 12:26:11.268547 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_deploy_prep.yml 2026-01-21 12:26:11.268563 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_kuttl.yml 2026-01-21 12:26:11.268578 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_kuttl_run.yml 2026-01-21 12:26:11.268594 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_horizon_prep.yml 2026-01-21 12:26:11.268610 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_infra.yml 2026-01-21 12:26:11.268626 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_infra_cleanup.yml 2026-01-21 12:26:11.268642 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_infra_kuttl.yml 2026-01-21 12:26:11.268658 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_infra_kuttl_run.yml 2026-01-21 12:26:11.268716 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_infra_prep.yml 2026-01-21 12:26:11.268750 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_input.yml 2026-01-21 12:26:11.268768 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_input_cleanup.yml 2026-01-21 12:26:11.268785 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab.yml 2026-01-21 12:26:11.268803 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_cleanup.yml 2026-01-21 12:26:11.268820 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_nat64_router.yml 2026-01-21 12:26:11.268836 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_nat64_router_cleanup.yml 2026-01-21 12:26:11.268853 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_network.yml 2026-01-21 12:26:11.268870 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_network_cleanup.yml 2026-01-21 12:26:11.268896 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_sno.yml 2026-01-21 12:26:11.268914 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ipv6_lab_sno_cleanup.yml 2026-01-21 12:26:11.268931 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic.yml 2026-01-21 12:26:11.268948 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_cleanup.yml 2026-01-21 12:26:11.268965 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_deploy.yml 2026-01-21 12:26:11.268982 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_deploy_cleanup.yml 2026-01-21 12:26:11.268999 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_deploy_prep.yml 2026-01-21 12:26:11.269015 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_kuttl.yml 2026-01-21 12:26:11.269032 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_kuttl_crc.yml 2026-01-21 12:26:11.269048 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_kuttl_run.yml 2026-01-21 12:26:11.269064 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ironic_prep.yml 2026-01-21 12:26:11.269081 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone.yml 2026-01-21 12:26:11.269098 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_cleanup.yml 2026-01-21 12:26:11.269114 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_deploy.yml 2026-01-21 12:26:11.269131 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_deploy_cleanup.yml 2026-01-21 12:26:11.269147 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_deploy_prep.yml 2026-01-21 12:26:11.269163 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_kuttl.yml 2026-01-21 12:26:11.269180 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_kuttl_run.yml 2026-01-21 12:26:11.269196 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_keystone_prep.yml 2026-01-21 12:26:11.269212 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_kuttl_common_cleanup.yml 2026-01-21 12:26:11.269228 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_kuttl_common_prep.yml 2026-01-21 12:26:11.269245 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_kuttl_db_cleanup.yml 2026-01-21 12:26:11.269261 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_kuttl_db_prep.yml 2026-01-21 12:26:11.269277 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_loki.yml 2026-01-21 12:26:11.269293 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_loki_cleanup.yml 2026-01-21 12:26:11.269309 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_loki_deploy.yml 2026-01-21 12:26:11.269325 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_loki_deploy_cleanup.yml 2026-01-21 12:26:11.269340 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_lvms.yml 2026-01-21 12:26:11.269356 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila.yml 2026-01-21 12:26:11.269372 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_cleanup.yml 2026-01-21 12:26:11.269392 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_deploy.yml 2026-01-21 12:26:11.269408 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_deploy_cleanup.yml 2026-01-21 12:26:11.269424 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_deploy_prep.yml 2026-01-21 12:26:11.269440 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_kuttl.yml 2026-01-21 12:26:11.269456 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_kuttl_run.yml 2026-01-21 12:26:11.269472 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_manila_prep.yml 2026-01-21 12:26:11.269488 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb.yml 2026-01-21 12:26:11.269504 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_chainsaw.yml 2026-01-21 12:26:11.269520 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_chainsaw_run.yml 2026-01-21 12:26:11.269536 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_cleanup.yml 2026-01-21 12:26:11.269551 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_deploy.yml 2026-01-21 12:26:11.269567 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_deploy_cleanup.yml 2026-01-21 12:26:11.269583 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_deploy_prep.yml 2026-01-21 12:26:11.269599 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_kuttl.yml 2026-01-21 12:26:11.269615 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_mariadb_kuttl_run.yml 2026-01-21 12:26:11.269630 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_memcached_deploy.yml 2026-01-21 12:26:11.269647 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_memcached_deploy_cleanup.yml 2026-01-21 12:26:11.269663 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_memcached_deploy_prep.yml 2026-01-21 12:26:11.269694 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_metallb.yml 2026-01-21 12:26:11.269714 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_metallb_cleanup.yml 2026-01-21 12:26:11.269731 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_metallb_config.yml 2026-01-21 12:26:11.269747 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_metallb_config_cleanup.yml 2026-01-21 12:26:11.269763 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_namespace.yml 2026-01-21 12:26:11.269779 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_namespace_cleanup.yml 2026-01-21 12:26:11.269795 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netattach.yml 2026-01-21 12:26:11.269811 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netattach_cleanup.yml 2026-01-21 12:26:11.269827 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netconfig_deploy.yml 2026-01-21 12:26:11.269844 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netconfig_deploy_cleanup.yml 2026-01-21 12:26:11.269863 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netconfig_deploy_prep.yml 2026-01-21 12:26:11.269886 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netobserv.yml 2026-01-21 12:26:11.269903 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netobserv_cleanup.yml 2026-01-21 12:26:11.269920 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netobserv_deploy.yml 2026-01-21 12:26:11.269937 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_netobserv_deploy_cleanup.yml 2026-01-21 12:26:11.269953 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_network_isolation_bridge.yml 2026-01-21 12:26:11.269970 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_network_isolation_bridge_cleanup.yml 2026-01-21 12:26:11.269986 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron.yml 2026-01-21 12:26:11.270002 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_cleanup.yml 2026-01-21 12:26:11.270019 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_deploy.yml 2026-01-21 12:26:11.270035 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_deploy_cleanup.yml 2026-01-21 12:26:11.270051 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_deploy_prep.yml 2026-01-21 12:26:11.270067 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_kuttl.yml 2026-01-21 12:26:11.270083 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_kuttl_run.yml 2026-01-21 12:26:11.270099 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_neutron_prep.yml 2026-01-21 12:26:11.270116 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nfs.yml 2026-01-21 12:26:11.270132 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nfs_cleanup.yml 2026-01-21 12:26:11.270148 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nmstate.yml 2026-01-21 12:26:11.270165 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nncp.yml 2026-01-21 12:26:11.270182 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nncp_cleanup.yml 2026-01-21 12:26:11.270198 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nova.yml 2026-01-21 12:26:11.270214 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nova_cleanup.yml 2026-01-21 12:26:11.270230 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nova_deploy.yml 2026-01-21 12:26:11.270246 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nova_deploy_cleanup.yml 2026-01-21 12:26:11.270263 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nova_deploy_prep.yml 2026-01-21 12:26:11.270279 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_nova_prep.yml 2026-01-21 12:26:11.270295 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia.yml 2026-01-21 12:26:11.270311 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_cleanup.yml 2026-01-21 12:26:11.270327 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_deploy.yml 2026-01-21 12:26:11.270344 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_deploy_cleanup.yml 2026-01-21 12:26:11.270363 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_deploy_prep.yml 2026-01-21 12:26:11.270380 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_kuttl.yml 2026-01-21 12:26:11.270396 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_kuttl_run.yml 2026-01-21 12:26:11.270412 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_octavia_prep.yml 2026-01-21 12:26:11.270428 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack.yml 2026-01-21 12:26:11.270444 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_cleanup.yml 2026-01-21 12:26:11.270460 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_crds.yml 2026-01-21 12:26:11.270476 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_crds_cleanup.yml 2026-01-21 12:26:11.270492 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_deploy.yml 2026-01-21 12:26:11.270508 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_deploy_cleanup.yml 2026-01-21 12:26:11.270524 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_deploy_prep.yml 2026-01-21 12:26:11.270540 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_init.yml 2026-01-21 12:26:11.270555 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_kuttl.yml 2026-01-21 12:26:11.270571 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_kuttl_run.yml 2026-01-21 12:26:11.270587 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_patch_version.yml 2026-01-21 12:26:11.270603 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_prep.yml 2026-01-21 12:26:11.270619 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_repo.yml 2026-01-21 12:26:11.270635 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_update_run.yml 2026-01-21 12:26:11.270651 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_wait.yml 2026-01-21 12:26:11.270667 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_openstack_wait_deploy.yml 2026-01-21 12:26:11.270701 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_operator_namespace.yml 2026-01-21 12:26:11.270718 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn.yml 2026-01-21 12:26:11.270735 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_cleanup.yml 2026-01-21 12:26:11.270751 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_deploy.yml 2026-01-21 12:26:11.270767 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_deploy_cleanup.yml 2026-01-21 12:26:11.270783 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_deploy_prep.yml 2026-01-21 12:26:11.270799 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_kuttl.yml 2026-01-21 12:26:11.270815 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_kuttl_run.yml 2026-01-21 12:26:11.270832 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_ovn_prep.yml 2026-01-21 12:26:11.270848 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement.yml 2026-01-21 12:26:11.270867 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_cleanup.yml 2026-01-21 12:26:11.270884 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_deploy.yml 2026-01-21 12:26:11.270899 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_deploy_cleanup.yml 2026-01-21 12:26:11.270915 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_deploy_prep.yml 2026-01-21 12:26:11.270930 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_kuttl.yml 2026-01-21 12:26:11.270945 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_kuttl_run.yml 2026-01-21 12:26:11.270961 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_placement_prep.yml 2026-01-21 12:26:11.270976 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rabbitmq.yml 2026-01-21 12:26:11.270991 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rabbitmq_cleanup.yml 2026-01-21 12:26:11.271007 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rabbitmq_deploy.yml 2026-01-21 12:26:11.271023 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rabbitmq_deploy_cleanup.yml 2026-01-21 12:26:11.271040 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rabbitmq_deploy_prep.yml 2026-01-21 12:26:11.271056 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rabbitmq_prep.yml 2026-01-21 12:26:11.271073 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_redis_deploy.yml 2026-01-21 12:26:11.271090 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_redis_deploy_cleanup.yml 2026-01-21 12:26:11.271106 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_redis_deploy_prep.yml 2026-01-21 12:26:11.271123 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rook.yml 2026-01-21 12:26:11.271291 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rook_cleanup.yml 2026-01-21 12:26:11.271323 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rook_crc_disk.yml 2026-01-21 12:26:11.271342 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rook_deploy.yml 2026-01-21 12:26:11.271359 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rook_deploy_prep.yml 2026-01-21 12:26:11.271376 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_rook_prep.yml 2026-01-21 12:26:11.271392 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_set_slower_etcd_profile.yml 2026-01-21 12:26:11.271409 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_standalone.yml 2026-01-21 12:26:11.271425 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_standalone_cleanup.yml 2026-01-21 12:26:11.271441 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_standalone_deploy.yml 2026-01-21 12:26:11.271457 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_standalone_revert.yml 2026-01-21 12:26:11.271473 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_standalone_snapshot.yml 2026-01-21 12:26:11.271490 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_standalone_sync.yml 2026-01-21 12:26:11.271510 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift.yml 2026-01-21 12:26:11.271528 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_cleanup.yml 2026-01-21 12:26:11.271545 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_deploy.yml 2026-01-21 12:26:11.271561 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_deploy_cleanup.yml 2026-01-21 12:26:11.271578 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_deploy_prep.yml 2026-01-21 12:26:11.271594 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_kuttl.yml 2026-01-21 12:26:11.271611 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_kuttl_run.yml 2026-01-21 12:26:11.271627 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_swift_prep.yml 2026-01-21 12:26:11.271643 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry.yml 2026-01-21 12:26:11.271659 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_cleanup.yml 2026-01-21 12:26:11.271690 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_deploy.yml 2026-01-21 12:26:11.271713 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_deploy_cleanup.yml 2026-01-21 12:26:11.271731 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_deploy_prep.yml 2026-01-21 12:26:11.271747 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_kuttl.yml 2026-01-21 12:26:11.271763 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_kuttl_run.yml 2026-01-21 12:26:11.271780 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_telemetry_prep.yml 2026-01-21 12:26:11.271796 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_tripleo_deploy.yml 2026-01-21 12:26:11.271812 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_update_services.yml 2026-01-21 12:26:11.271829 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_update_system.yml 2026-01-21 12:26:11.271980 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_validate_marketplace.yml 2026-01-21 12:26:11.272006 | controller | >f+++++++++ ci-framework-data/artifacts/roles/install_yamls_makes/tasks/make_wait.yml 2026-01-21 12:26:11.272027 | controller | cd+++++++++ ci-framework-data/artifacts/yum_repos/ 2026-01-21 12:26:11.272064 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/delorean-antelope-testing.repo 2026-01-21 12:26:11.272082 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/delorean.repo 2026-01-21 12:26:11.272100 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/delorean.repo.md5 2026-01-21 12:26:11.272270 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/repo-setup-centos-appstream.repo 2026-01-21 12:26:11.272547 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/repo-setup-centos-baseos.repo 2026-01-21 12:26:11.272569 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/repo-setup-centos-highavailability.repo 2026-01-21 12:26:11.272588 | controller | >f+++++++++ ci-framework-data/artifacts/yum_repos/repo-setup-centos-powertools.repo 2026-01-21 12:26:11.272606 | controller | cd+++++++++ ci-framework-data/logs/ 2026-01-21 12:26:11.272624 | controller | >f+++++++++ ci-framework-data/logs/ansible.log 2026-01-21 12:26:11.272642 | controller | >f+++++++++ ci-framework-data/logs/ci_script_000_copy_logs_from_crc.log 2026-01-21 12:26:11.272664 | controller | >f+++++++++ ci-framework-data/logs/ci_script_000_prepare_root_ssh.log 2026-01-21 12:26:11.272703 | controller | >f+++++++++ ci-framework-data/logs/ci_script_000_run_download.log 2026-01-21 12:26:11.272722 | controller | >f+++++++++ ci-framework-data/logs/ci_script_000_run_openstack_must_gather.log 2026-01-21 12:26:11.272738 | controller | >f+++++++++ ci-framework-data/logs/ci_script_001_run_hook_without_retry.log 2026-01-21 12:26:11.272755 | controller | >f+++++++++ ci-framework-data/logs/ci_script_002_fetch_openshift.log 2026-01-21 12:26:11.272771 | controller | >f+++++++++ ci-framework-data/logs/ci_script_003_run_openstack.log 2026-01-21 12:26:11.272790 | controller | >f+++++++++ ci-framework-data/logs/ci_script_004_run_hook_without_retry_deploy.log 2026-01-21 12:26:11.272809 | controller | >f+++++++++ ci-framework-data/logs/ci_script_005_stop_watcher.log 2026-01-21 12:26:11.272825 | controller | >f+++++++++ ci-framework-data/logs/ci_script_006_install_watcher_operator_in.log 2026-01-21 12:26:11.272840 | controller | >f+++++++++ ci-framework-data/logs/post_install_operators_kuttl_from_operator_deploy_watcher_operator.log 2026-01-21 12:26:11.272857 | controller | >f+++++++++ ci-framework-data/logs/pre_infra_download_needed_tools.log 2026-01-21 12:26:11.272873 | controller | cd+++++++++ ci-framework-data/logs/2026-01-21_11-38/ 2026-01-21 12:26:11.272889 | controller | >f+++++++++ ci-framework-data/logs/2026-01-21_11-38/ansible.log 2026-01-21 12:26:11.272905 | controller | cd+++++++++ ci-framework-data/logs/crc/ 2026-01-21 12:26:11.273048 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/ 2026-01-21 12:26:11.273077 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/ 2026-01-21 12:26:11.273097 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager-operator_cert-manager-operator-controller-manager-64cf6dff88-dc7jl_c784a907-759a-4d6c-8c04-4254fa1f0b00/ 2026-01-21 12:26:11.273116 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager-operator_cert-manager-operator-controller-manager-64cf6dff88-dc7jl_c784a907-759a-4d6c-8c04-4254fa1f0b00/cert-manager-operator/ 2026-01-21 12:26:11.273135 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager-operator_cert-manager-operator-controller-manager-64cf6dff88-dc7jl_c784a907-759a-4d6c-8c04-4254fa1f0b00/cert-manager-operator/0.log 2026-01-21 12:26:11.273153 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-86cb77c54b-946sh_d5dc9762-e122-475f-a1a2-2d9711313716/ 2026-01-21 12:26:11.273171 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-86cb77c54b-946sh_d5dc9762-e122-475f-a1a2-2d9711313716/cert-manager-controller/ 2026-01-21 12:26:11.273188 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-86cb77c54b-946sh_d5dc9762-e122-475f-a1a2-2d9711313716/cert-manager-controller/0.log 2026-01-21 12:26:11.273204 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-cainjector-855d9ccff4-gl7pg_1ae434a0-174f-4c93-bf2c-ab2091c54e6c/ 2026-01-21 12:26:11.273221 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-cainjector-855d9ccff4-gl7pg_1ae434a0-174f-4c93-bf2c-ab2091c54e6c/cert-manager-cainjector/ 2026-01-21 12:26:11.273237 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-cainjector-855d9ccff4-gl7pg_1ae434a0-174f-4c93-bf2c-ab2091c54e6c/cert-manager-cainjector/0.log 2026-01-21 12:26:11.273254 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-webhook-f4fb5df64-9vbqj_d2372e44-af17-4c55-9a11-67fb28adcc08/ 2026-01-21 12:26:11.273270 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-webhook-f4fb5df64-9vbqj_d2372e44-af17-4c55-9a11-67fb28adcc08/cert-manager-webhook/ 2026-01-21 12:26:11.273291 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/cert-manager_cert-manager-webhook-f4fb5df64-9vbqj_d2372e44-af17-4c55-9a11-67fb28adcc08/cert-manager-webhook/0.log 2026-01-21 12:26:11.273308 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/ 2026-01-21 12:26:11.273324 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/csi-provisioner/ 2026-01-21 12:26:11.273340 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/csi-provisioner/0.log 2026-01-21 12:26:11.273357 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/hostpath-provisioner/ 2026-01-21 12:26:11.273374 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/hostpath-provisioner/0.log 2026-01-21 12:26:11.273392 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/liveness-probe/ 2026-01-21 12:26:11.273408 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/liveness-probe/0.log 2026-01-21 12:26:11.273425 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/node-driver-registrar/ 2026-01-21 12:26:11.273442 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/hostpath-provisioner_csi-hostpathplugin-qrrl6_15dcf9e9-44e8-4662-9f3d-6cef771808c5/node-driver-registrar/0.log 2026-01-21 12:26:11.273458 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_controller-6968d8fdc4-q24bt_66dc8772-25c5-4ad1-b0fa-6981e3158ad5/ 2026-01-21 12:26:11.273475 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_controller-6968d8fdc4-q24bt_66dc8772-25c5-4ad1-b0fa-6981e3158ad5/controller/ 2026-01-21 12:26:11.273491 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_controller-6968d8fdc4-q24bt_66dc8772-25c5-4ad1-b0fa-6981e3158ad5/controller/0.log.gz 2026-01-21 12:26:11.273508 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_controller-6968d8fdc4-q24bt_66dc8772-25c5-4ad1-b0fa-6981e3158ad5/kube-rbac-proxy/ 2026-01-21 12:26:11.273524 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_controller-6968d8fdc4-q24bt_66dc8772-25c5-4ad1-b0fa-6981e3158ad5/kube-rbac-proxy/0.log 2026-01-21 12:26:11.273541 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/ 2026-01-21 12:26:11.273557 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/0715d24d3722a214d53aa5c1c96e4d34401db3d24427566917ac647850429e06.log 2026-01-21 12:26:11.273573 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/controller/ 2026-01-21 12:26:11.273590 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/controller/0.log 2026-01-21 12:26:11.273606 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/cp-frr-files/ 2026-01-21 12:26:11.273626 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/cp-frr-files/0.log 2026-01-21 12:26:11.273643 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/cp-metrics/ 2026-01-21 12:26:11.273659 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/cp-metrics/0.log 2026-01-21 12:26:11.273690 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/cp-reloader/ 2026-01-21 12:26:11.273711 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/cp-reloader/0.log 2026-01-21 12:26:11.273728 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/frr-metrics/ 2026-01-21 12:26:11.273745 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/frr-metrics/0.log 2026-01-21 12:26:11.273761 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/frr/ 2026-01-21 12:26:11.273778 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/frr/0.log.gz 2026-01-21 12:26:11.273794 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/kube-rbac-proxy-frr/ 2026-01-21 12:26:11.273810 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/kube-rbac-proxy-frr/0.log 2026-01-21 12:26:11.273827 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/kube-rbac-proxy/ 2026-01-21 12:26:11.273843 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/kube-rbac-proxy/0.log 2026-01-21 12:26:11.273860 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/reloader/ 2026-01-21 12:26:11.273876 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-7tz4m_9449246f-d4a0-407f-8e9f-cb7271c90d72/reloader/0.log 2026-01-21 12:26:11.273892 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-webhook-server-7df86c4f6c-9d94c_0b7695ad-2b58-4be9-911d-bc83bece0db7/ 2026-01-21 12:26:11.273908 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-webhook-server-7df86c4f6c-9d94c_0b7695ad-2b58-4be9-911d-bc83bece0db7/frr-k8s-webhook-server/ 2026-01-21 12:26:11.273924 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_frr-k8s-webhook-server-7df86c4f6c-9d94c_0b7695ad-2b58-4be9-911d-bc83bece0db7/frr-k8s-webhook-server/0.log 2026-01-21 12:26:11.273941 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_metallb-operator-controller-manager-57547767ff-zrxjk_1f8eea58-9366-4bb1-a9d2-dc8842674dc2/ 2026-01-21 12:26:11.273957 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_metallb-operator-controller-manager-57547767ff-zrxjk_1f8eea58-9366-4bb1-a9d2-dc8842674dc2/manager/ 2026-01-21 12:26:11.273973 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_metallb-operator-controller-manager-57547767ff-zrxjk_1f8eea58-9366-4bb1-a9d2-dc8842674dc2/manager/0.log 2026-01-21 12:26:11.273993 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_metallb-operator-webhook-server-5b9dd8b59d-59895_955477b1-b9f0-41a2-aa5b-2e2f47495422/ 2026-01-21 12:26:11.274010 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_metallb-operator-webhook-server-5b9dd8b59d-59895_955477b1-b9f0-41a2-aa5b-2e2f47495422/webhook-server/ 2026-01-21 12:26:11.274027 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_metallb-operator-webhook-server-5b9dd8b59d-59895_955477b1-b9f0-41a2-aa5b-2e2f47495422/webhook-server/0.log 2026-01-21 12:26:11.274043 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_speaker-zxq6z_0ac5019d-ffb4-4cb6-9042-1b983b15841a/ 2026-01-21 12:26:11.274060 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_speaker-zxq6z_0ac5019d-ffb4-4cb6-9042-1b983b15841a/kube-rbac-proxy/ 2026-01-21 12:26:11.274076 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_speaker-zxq6z_0ac5019d-ffb4-4cb6-9042-1b983b15841a/kube-rbac-proxy/0.log 2026-01-21 12:26:11.274093 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_speaker-zxq6z_0ac5019d-ffb4-4cb6-9042-1b983b15841a/speaker/ 2026-01-21 12:26:11.274120 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/metallb-system_speaker-zxq6z_0ac5019d-ffb4-4cb6-9042-1b983b15841a/speaker/0.log.gz 2026-01-21 12:26:11.274138 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver-operator_openshift-apiserver-operator-796bbdcf4f-77kwx_b9b96df7-05d7-4cd0-9e30-c5e485f31804/ 2026-01-21 12:26:11.274154 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver-operator_openshift-apiserver-operator-796bbdcf4f-77kwx_b9b96df7-05d7-4cd0-9e30-c5e485f31804/openshift-apiserver-operator/ 2026-01-21 12:26:11.274171 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver-operator_openshift-apiserver-operator-796bbdcf4f-77kwx_b9b96df7-05d7-4cd0-9e30-c5e485f31804/openshift-apiserver-operator/0.log 2026-01-21 12:26:11.274187 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/ 2026-01-21 12:26:11.274205 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/fix-audit-permissions/ 2026-01-21 12:26:11.274221 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/fix-audit-permissions/0.log 2026-01-21 12:26:11.274237 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/openshift-apiserver-check-endpoints/ 2026-01-21 12:26:11.274253 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/openshift-apiserver-check-endpoints/0.log 2026-01-21 12:26:11.274269 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/openshift-apiserver/ 2026-01-21 12:26:11.274286 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-apiserver_apiserver-76f77b778f-pxkk7_e0376dda-f02a-464e-ae41-18d6fddd7097/openshift-apiserver/0.log 2026-01-21 12:26:11.274303 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-authentication-operator_authentication-operator-69f744f599-4kjnr_7cbece9b-282a-4634-b41d-85f872a5be93/ 2026-01-21 12:26:11.274327 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-authentication-operator_authentication-operator-69f744f599-4kjnr_7cbece9b-282a-4634-b41d-85f872a5be93/authentication-operator/ 2026-01-21 12:26:11.274344 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-authentication-operator_authentication-operator-69f744f599-4kjnr_7cbece9b-282a-4634-b41d-85f872a5be93/authentication-operator/0.log 2026-01-21 12:26:11.274361 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-authentication_oauth-openshift-7445b77485-m7ckc_e9b8abff-ca3c-47d7-9c5e-f6a23574e25d/ 2026-01-21 12:26:11.274378 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-authentication_oauth-openshift-7445b77485-m7ckc_e9b8abff-ca3c-47d7-9c5e-f6a23574e25d/oauth-openshift/ 2026-01-21 12:26:11.274394 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-authentication_oauth-openshift-7445b77485-m7ckc_e9b8abff-ca3c-47d7-9c5e-f6a23574e25d/oauth-openshift/0.log 2026-01-21 12:26:11.274411 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-machine-approver_machine-approver-56656f9798-g66gm_81742698-2b71-46b2-93fa-1552cfa27f8a/ 2026-01-21 12:26:11.274550 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-machine-approver_machine-approver-56656f9798-g66gm_81742698-2b71-46b2-93fa-1552cfa27f8a/kube-rbac-proxy/ 2026-01-21 12:26:11.274576 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-machine-approver_machine-approver-56656f9798-g66gm_81742698-2b71-46b2-93fa-1552cfa27f8a/kube-rbac-proxy/0.log 2026-01-21 12:26:11.274595 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-machine-approver_machine-approver-56656f9798-g66gm_81742698-2b71-46b2-93fa-1552cfa27f8a/machine-approver-controller/ 2026-01-21 12:26:11.274612 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-machine-approver_machine-approver-56656f9798-g66gm_81742698-2b71-46b2-93fa-1552cfa27f8a/machine-approver-controller/0.log 2026-01-21 12:26:11.274629 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-samples-operator_cluster-samples-operator-665b6dd947-gqsng_b97d7bcf-6463-4a55-a0f6-ebe0f3bda8e9/ 2026-01-21 12:26:11.274646 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-samples-operator_cluster-samples-operator-665b6dd947-gqsng_b97d7bcf-6463-4a55-a0f6-ebe0f3bda8e9/6c240036bcbf63da0ee6c1c543c2c01396457f727d9e8215bdef1cc338163149.log 2026-01-21 12:26:11.274663 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-samples-operator_cluster-samples-operator-665b6dd947-gqsng_b97d7bcf-6463-4a55-a0f6-ebe0f3bda8e9/cluster-samples-operator-watch/ 2026-01-21 12:26:11.274699 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-samples-operator_cluster-samples-operator-665b6dd947-gqsng_b97d7bcf-6463-4a55-a0f6-ebe0f3bda8e9/cluster-samples-operator-watch/0.log 2026-01-21 12:26:11.274718 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-samples-operator_cluster-samples-operator-665b6dd947-gqsng_b97d7bcf-6463-4a55-a0f6-ebe0f3bda8e9/cluster-samples-operator/ 2026-01-21 12:26:11.274736 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-samples-operator_cluster-samples-operator-665b6dd947-gqsng_b97d7bcf-6463-4a55-a0f6-ebe0f3bda8e9/cluster-samples-operator/0.log 2026-01-21 12:26:11.274753 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-version_cluster-version-operator-5c965bbfc6-t6bnk_1b1fd621-6a72-4e19-b778-3a8e3950ee68/ 2026-01-21 12:26:11.274769 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-version_cluster-version-operator-5c965bbfc6-t6bnk_1b1fd621-6a72-4e19-b778-3a8e3950ee68/cluster-version-operator/ 2026-01-21 12:26:11.274790 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-cluster-version_cluster-version-operator-5c965bbfc6-t6bnk_1b1fd621-6a72-4e19-b778-3a8e3950ee68/cluster-version-operator/0.log.gz 2026-01-21 12:26:11.274807 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-config-operator_openshift-config-operator-7777fb866f-9fbhr_3dddbbd4-eb3f-436d-8c53-cf413cecca31/ 2026-01-21 12:26:11.274823 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-config-operator_openshift-config-operator-7777fb866f-9fbhr_3dddbbd4-eb3f-436d-8c53-cf413cecca31/openshift-api/ 2026-01-21 12:26:11.274840 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-config-operator_openshift-config-operator-7777fb866f-9fbhr_3dddbbd4-eb3f-436d-8c53-cf413cecca31/openshift-api/0.log 2026-01-21 12:26:11.274856 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-config-operator_openshift-config-operator-7777fb866f-9fbhr_3dddbbd4-eb3f-436d-8c53-cf413cecca31/openshift-config-operator/ 2026-01-21 12:26:11.274873 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-config-operator_openshift-config-operator-7777fb866f-9fbhr_3dddbbd4-eb3f-436d-8c53-cf413cecca31/openshift-config-operator/0.log 2026-01-21 12:26:11.274889 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console-operator_console-operator-58897d9998-4d577_78713d9a-139c-4d4e-8068-a0d0d98b86df/ 2026-01-21 12:26:11.274905 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console-operator_console-operator-58897d9998-4d577_78713d9a-139c-4d4e-8068-a0d0d98b86df/console-operator/ 2026-01-21 12:26:11.274922 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console-operator_console-operator-58897d9998-4d577_78713d9a-139c-4d4e-8068-a0d0d98b86df/console-operator/0.log 2026-01-21 12:26:11.274938 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_console-64479565b-fzhdc_4c315b0b-2610-4eb5-822a-2cef565890d1/ 2026-01-21 12:26:11.274955 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_console-64479565b-fzhdc_4c315b0b-2610-4eb5-822a-2cef565890d1/console/ 2026-01-21 12:26:11.274972 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_console-64479565b-fzhdc_4c315b0b-2610-4eb5-822a-2cef565890d1/console/0.log 2026-01-21 12:26:11.274988 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_downloads-7954f5f757-vw8cb_890e3b6e-bd8d-438c-992b-508bb751bdca/ 2026-01-21 12:26:11.275004 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_downloads-7954f5f757-vw8cb_890e3b6e-bd8d-438c-992b-508bb751bdca/download-server/ 2026-01-21 12:26:11.275021 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_downloads-7954f5f757-vw8cb_890e3b6e-bd8d-438c-992b-508bb751bdca/download-server/2.log 2026-01-21 12:26:11.275037 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-console_downloads-7954f5f757-vw8cb_890e3b6e-bd8d-438c-992b-508bb751bdca/download-server/3.log 2026-01-21 12:26:11.275053 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-controller-manager-operator_openshift-controller-manager-operator-756b6f6bc6-jqrcn_5cb8f784-0f6a-43c7-a37a-d7f65668af7a/ 2026-01-21 12:26:11.275069 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-controller-manager-operator_openshift-controller-manager-operator-756b6f6bc6-jqrcn_5cb8f784-0f6a-43c7-a37a-d7f65668af7a/openshift-controller-manager-operator/ 2026-01-21 12:26:11.275089 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-controller-manager-operator_openshift-controller-manager-operator-756b6f6bc6-jqrcn_5cb8f784-0f6a-43c7-a37a-d7f65668af7a/openshift-controller-manager-operator/0.log 2026-01-21 12:26:11.275106 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-controller-manager_controller-manager-58dccb64f7-7hwmg_2909711b-8468-4125-977d-c9c64668ed22/ 2026-01-21 12:26:11.275123 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-controller-manager_controller-manager-58dccb64f7-7hwmg_2909711b-8468-4125-977d-c9c64668ed22/controller-manager/ 2026-01-21 12:26:11.275139 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-controller-manager_controller-manager-58dccb64f7-7hwmg_2909711b-8468-4125-977d-c9c64668ed22/controller-manager/0.log 2026-01-21 12:26:11.275156 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns-operator_dns-operator-744455d44c-kk7wd_be500af9-e814-41db-be2a-e4f3fa9d46bb/ 2026-01-21 12:26:11.275172 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns-operator_dns-operator-744455d44c-kk7wd_be500af9-e814-41db-be2a-e4f3fa9d46bb/dns-operator/ 2026-01-21 12:26:11.275188 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns-operator_dns-operator-744455d44c-kk7wd_be500af9-e814-41db-be2a-e4f3fa9d46bb/dns-operator/0.log 2026-01-21 12:26:11.275205 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns-operator_dns-operator-744455d44c-kk7wd_be500af9-e814-41db-be2a-e4f3fa9d46bb/kube-rbac-proxy/ 2026-01-21 12:26:11.275221 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns-operator_dns-operator-744455d44c-kk7wd_be500af9-e814-41db-be2a-e4f3fa9d46bb/kube-rbac-proxy/0.log 2026-01-21 12:26:11.275237 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_dns-default-vz4nw_6b7fdcdd-7176-41a4-a4a2-3a2db9c672c6/ 2026-01-21 12:26:11.275253 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_dns-default-vz4nw_6b7fdcdd-7176-41a4-a4a2-3a2db9c672c6/dns/ 2026-01-21 12:26:11.275269 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_dns-default-vz4nw_6b7fdcdd-7176-41a4-a4a2-3a2db9c672c6/dns/0.log 2026-01-21 12:26:11.275285 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_dns-default-vz4nw_6b7fdcdd-7176-41a4-a4a2-3a2db9c672c6/kube-rbac-proxy/ 2026-01-21 12:26:11.275300 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_dns-default-vz4nw_6b7fdcdd-7176-41a4-a4a2-3a2db9c672c6/kube-rbac-proxy/0.log 2026-01-21 12:26:11.275317 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_node-resolver-rzmbp_ae658aae-64a2-4df8-938c-7a4c2a35655f/ 2026-01-21 12:26:11.275333 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_node-resolver-rzmbp_ae658aae-64a2-4df8-938c-7a4c2a35655f/dns-node-resolver/ 2026-01-21 12:26:11.275349 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-dns_node-resolver-rzmbp_ae658aae-64a2-4df8-938c-7a4c2a35655f/dns-node-resolver/0.log 2026-01-21 12:26:11.275365 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd-operator_etcd-operator-b45778765-b7tzs_cfd1ca7f-583a-40a7-a485-fb01c60d77c6/ 2026-01-21 12:26:11.275382 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd-operator_etcd-operator-b45778765-b7tzs_cfd1ca7f-583a-40a7-a485-fb01c60d77c6/etcd-operator/ 2026-01-21 12:26:11.275398 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd-operator_etcd-operator-b45778765-b7tzs_cfd1ca7f-583a-40a7-a485-fb01c60d77c6/etcd-operator/0.log 2026-01-21 12:26:11.275418 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/ 2026-01-21 12:26:11.275435 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-ensure-env-vars/ 2026-01-21 12:26:11.275451 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-ensure-env-vars/0.log 2026-01-21 12:26:11.275467 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-metrics/ 2026-01-21 12:26:11.275483 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-metrics/0.log 2026-01-21 12:26:11.275499 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-readyz/ 2026-01-21 12:26:11.275514 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-readyz/0.log 2026-01-21 12:26:11.275530 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-resources-copy/ 2026-01-21 12:26:11.275546 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-resources-copy/0.log 2026-01-21 12:26:11.275562 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-rev/ 2026-01-21 12:26:11.275579 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd-rev/0.log 2026-01-21 12:26:11.275595 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd/ 2026-01-21 12:26:11.275612 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcd/0.log 2026-01-21 12:26:11.275629 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcdctl/ 2026-01-21 12:26:11.275645 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/etcdctl/0.log 2026-01-21 12:26:11.275661 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/setup/ 2026-01-21 12:26:11.275692 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-etcd_etcd-crc_2139d3e2895fc6797b9c76a1b4c9886d/setup/0.log 2026-01-21 12:26:11.275714 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_cluster-image-registry-operator-dc59b4c8b-cq2j2_5ad56032-3192-4b42-b7ca-a8c3bac978d1/ 2026-01-21 12:26:11.275732 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_cluster-image-registry-operator-dc59b4c8b-cq2j2_5ad56032-3192-4b42-b7ca-a8c3bac978d1/cluster-image-registry-operator/ 2026-01-21 12:26:11.275749 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_cluster-image-registry-operator-dc59b4c8b-cq2j2_5ad56032-3192-4b42-b7ca-a8c3bac978d1/cluster-image-registry-operator/0.log 2026-01-21 12:26:11.275766 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_image-registry-66df7c8f76-dfq6r_191ca2d9-4f66-4f3e-b27d-bac48d1f150a/ 2026-01-21 12:26:11.275783 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_image-registry-66df7c8f76-dfq6r_191ca2d9-4f66-4f3e-b27d-bac48d1f150a/registry/ 2026-01-21 12:26:11.275804 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_image-registry-66df7c8f76-dfq6r_191ca2d9-4f66-4f3e-b27d-bac48d1f150a/registry/0.log 2026-01-21 12:26:11.275821 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_node-ca-jqsxs_4e986f25-2ad4-428d-b6a5-f99e1a480285/ 2026-01-21 12:26:11.275837 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_node-ca-jqsxs_4e986f25-2ad4-428d-b6a5-f99e1a480285/node-ca/ 2026-01-21 12:26:11.275854 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-image-registry_node-ca-jqsxs_4e986f25-2ad4-428d-b6a5-f99e1a480285/node-ca/0.log 2026-01-21 12:26:11.275871 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-canary_ingress-canary-85pbp_0bccbd0b-3782-4ad2-bdd5-f21e5109165d/ 2026-01-21 12:26:11.275889 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-canary_ingress-canary-85pbp_0bccbd0b-3782-4ad2-bdd5-f21e5109165d/serve-healthcheck-canary/ 2026-01-21 12:26:11.275906 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-canary_ingress-canary-85pbp_0bccbd0b-3782-4ad2-bdd5-f21e5109165d/serve-healthcheck-canary/0.log 2026-01-21 12:26:11.275923 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-operator_ingress-operator-5b745b69d9-4kbqk_dfbafd02-2fcf-4a25-a454-ade91c336036/ 2026-01-21 12:26:11.275939 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-operator_ingress-operator-5b745b69d9-4kbqk_dfbafd02-2fcf-4a25-a454-ade91c336036/ingress-operator/ 2026-01-21 12:26:11.275955 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-operator_ingress-operator-5b745b69d9-4kbqk_dfbafd02-2fcf-4a25-a454-ade91c336036/ingress-operator/0.log 2026-01-21 12:26:11.275972 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-operator_ingress-operator-5b745b69d9-4kbqk_dfbafd02-2fcf-4a25-a454-ade91c336036/kube-rbac-proxy/ 2026-01-21 12:26:11.275988 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress-operator_ingress-operator-5b745b69d9-4kbqk_dfbafd02-2fcf-4a25-a454-ade91c336036/kube-rbac-proxy/0.log 2026-01-21 12:26:11.276005 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress_router-default-5444994796-n2k47_fb3fdc07-c6f5-4330-8b00-e454c98ef11d/ 2026-01-21 12:26:11.276021 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress_router-default-5444994796-n2k47_fb3fdc07-c6f5-4330-8b00-e454c98ef11d/router/ 2026-01-21 12:26:11.276066 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ingress_router-default-5444994796-n2k47_fb3fdc07-c6f5-4330-8b00-e454c98ef11d/router/0.log 2026-01-21 12:26:11.276083 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver-operator_kube-apiserver-operator-766d6c64bb-5ml9s_5dccce36-49ce-4eea-ac64-60faf9ba2e04/ 2026-01-21 12:26:11.276099 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver-operator_kube-apiserver-operator-766d6c64bb-5ml9s_5dccce36-49ce-4eea-ac64-60faf9ba2e04/kube-apiserver-operator/ 2026-01-21 12:26:11.276115 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver-operator_kube-apiserver-operator-766d6c64bb-5ml9s_5dccce36-49ce-4eea-ac64-60faf9ba2e04/kube-apiserver-operator/0.log 2026-01-21 12:26:11.276131 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_installer-9-crc_9358efee-87a3-49bf-a75c-a45dc2ac2987/ 2026-01-21 12:26:11.276147 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_installer-9-crc_9358efee-87a3-49bf-a75c-a45dc2ac2987/installer/ 2026-01-21 12:26:11.276168 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_installer-9-crc_9358efee-87a3-49bf-a75c-a45dc2ac2987/installer/0.log 2026-01-21 12:26:11.276187 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/ 2026-01-21 12:26:11.276203 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-cert-regeneration-controller/ 2026-01-21 12:26:11.276219 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-cert-regeneration-controller/0.log 2026-01-21 12:26:11.276235 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-cert-syncer/ 2026-01-21 12:26:11.276251 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-cert-syncer/0.log 2026-01-21 12:26:11.276267 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-check-endpoints/ 2026-01-21 12:26:11.276283 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-check-endpoints/0.log 2026-01-21 12:26:11.276300 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-insecure-readyz/ 2026-01-21 12:26:11.276316 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver-insecure-readyz/0.log 2026-01-21 12:26:11.276332 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver/ 2026-01-21 12:26:11.276348 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/kube-apiserver/0.log 2026-01-21 12:26:11.276365 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/setup/ 2026-01-21 12:26:11.276381 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_kube-apiserver-crc_71bb4a3aecc4ba5b26c4b7318770ce13/setup/0.log 2026-01-21 12:26:11.276397 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_revision-pruner-8-crc_334a23ee-2f20-4067-92a6-ff134cbd5bf2/ 2026-01-21 12:26:11.276414 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_revision-pruner-8-crc_334a23ee-2f20-4067-92a6-ff134cbd5bf2/pruner/ 2026-01-21 12:26:11.276430 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_revision-pruner-8-crc_334a23ee-2f20-4067-92a6-ff134cbd5bf2/pruner/0.log 2026-01-21 12:26:11.276446 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_revision-pruner-9-crc_f4d365b7-841c-44cb-ab4d-9fcb8493ffd7/ 2026-01-21 12:26:11.276462 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_revision-pruner-9-crc_f4d365b7-841c-44cb-ab4d-9fcb8493ffd7/pruner/ 2026-01-21 12:26:11.276478 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-apiserver_revision-pruner-9-crc_f4d365b7-841c-44cb-ab4d-9fcb8493ffd7/pruner/0.log 2026-01-21 12:26:11.276497 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager-operator_kube-controller-manager-operator-78b949d7b-28hkz_2ce7cc91-68ce-4bcc-99a0-436380c8a2e8/ 2026-01-21 12:26:11.276513 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager-operator_kube-controller-manager-operator-78b949d7b-28hkz_2ce7cc91-68ce-4bcc-99a0-436380c8a2e8/kube-controller-manager-operator/ 2026-01-21 12:26:11.276530 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager-operator_kube-controller-manager-operator-78b949d7b-28hkz_2ce7cc91-68ce-4bcc-99a0-436380c8a2e8/kube-controller-manager-operator/0.log 2026-01-21 12:26:11.276547 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/ 2026-01-21 12:26:11.276563 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/cluster-policy-controller/ 2026-01-21 12:26:11.276580 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/cluster-policy-controller/0.log 2026-01-21 12:26:11.276596 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager-cert-syncer/ 2026-01-21 12:26:11.276612 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager-cert-syncer/0.log 2026-01-21 12:26:11.276629 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager-recovery-controller/ 2026-01-21 12:26:11.276646 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager-recovery-controller/0.log 2026-01-21 12:26:11.276663 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager/ 2026-01-21 12:26:11.276708 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager/0.log 2026-01-21 12:26:11.276731 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_kube-controller-manager-crc_f614b9022728cf315e60c057852e563e/kube-controller-manager/1.log.gz 2026-01-21 12:26:11.276748 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_revision-pruner-9-crc_da6cdc16-92f1-4475-99f2-c087b77d76cf/ 2026-01-21 12:26:11.276765 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_revision-pruner-9-crc_da6cdc16-92f1-4475-99f2-c087b77d76cf/pruner/ 2026-01-21 12:26:11.276781 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-controller-manager_revision-pruner-9-crc_da6cdc16-92f1-4475-99f2-c087b77d76cf/pruner/0.log 2026-01-21 12:26:11.276798 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler-operator_openshift-kube-scheduler-operator-5fdd9b5758-jb9kj_c93dc177-affe-4232-9b28-fd8006418818/ 2026-01-21 12:26:11.276815 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler-operator_openshift-kube-scheduler-operator-5fdd9b5758-jb9kj_c93dc177-affe-4232-9b28-fd8006418818/kube-scheduler-operator-container/ 2026-01-21 12:26:11.276835 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler-operator_openshift-kube-scheduler-operator-5fdd9b5758-jb9kj_c93dc177-affe-4232-9b28-fd8006418818/kube-scheduler-operator-container/0.log 2026-01-21 12:26:11.276852 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/ 2026-01-21 12:26:11.276869 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/kube-scheduler-cert-syncer/ 2026-01-21 12:26:11.276886 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/kube-scheduler-cert-syncer/0.log 2026-01-21 12:26:11.276902 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/kube-scheduler-recovery-controller/ 2026-01-21 12:26:11.276917 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/kube-scheduler-recovery-controller/0.log 2026-01-21 12:26:11.276933 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/kube-scheduler/ 2026-01-21 12:26:11.276948 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/kube-scheduler/0.log 2026-01-21 12:26:11.276964 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/wait-for-host-port/ 2026-01-21 12:26:11.276979 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-scheduler_openshift-kube-scheduler-crc_3dcd261975c3d6b9a6ad6367fd4facd3/wait-for-host-port/0.log 2026-01-21 12:26:11.276994 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-b67b599dd-w2zsd_05b134e2-b96d-4a00-9681-6c8ce017bc74/ 2026-01-21 12:26:11.277010 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-b67b599dd-w2zsd_05b134e2-b96d-4a00-9681-6c8ce017bc74/kube-storage-version-migrator-operator/ 2026-01-21 12:26:11.277026 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-b67b599dd-w2zsd_05b134e2-b96d-4a00-9681-6c8ce017bc74/kube-storage-version-migrator-operator/0.log 2026-01-21 12:26:11.277043 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator_migrator-59844c95c7-hfk49_dffa6415-1a36-41b0-9919-a04bea0bdff8/ 2026-01-21 12:26:11.277060 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator_migrator-59844c95c7-hfk49_dffa6415-1a36-41b0-9919-a04bea0bdff8/graceful-termination/ 2026-01-21 12:26:11.277076 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator_migrator-59844c95c7-hfk49_dffa6415-1a36-41b0-9919-a04bea0bdff8/graceful-termination/0.log 2026-01-21 12:26:11.277092 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator_migrator-59844c95c7-hfk49_dffa6415-1a36-41b0-9919-a04bea0bdff8/migrator/ 2026-01-21 12:26:11.277112 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-kube-storage-version-migrator_migrator-59844c95c7-hfk49_dffa6415-1a36-41b0-9919-a04bea0bdff8/migrator/0.log 2026-01-21 12:26:11.277129 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_control-plane-machine-set-operator-78cbb6b69f-2fd99_5c2fa6a9-ee76-4308-a8f1-095d9720c688/ 2026-01-21 12:26:11.277145 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_control-plane-machine-set-operator-78cbb6b69f-2fd99_5c2fa6a9-ee76-4308-a8f1-095d9720c688/control-plane-machine-set-operator/ 2026-01-21 12:26:11.277161 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_control-plane-machine-set-operator-78cbb6b69f-2fd99_5c2fa6a9-ee76-4308-a8f1-095d9720c688/control-plane-machine-set-operator/0.log 2026-01-21 12:26:11.277178 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_machine-api-operator-5694c8668f-bmpxp_5a264bb6-3e63-4411-b0a4-95be21527653/ 2026-01-21 12:26:11.277194 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_machine-api-operator-5694c8668f-bmpxp_5a264bb6-3e63-4411-b0a4-95be21527653/kube-rbac-proxy/ 2026-01-21 12:26:11.277211 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_machine-api-operator-5694c8668f-bmpxp_5a264bb6-3e63-4411-b0a4-95be21527653/kube-rbac-proxy/0.log 2026-01-21 12:26:11.277227 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_machine-api-operator-5694c8668f-bmpxp_5a264bb6-3e63-4411-b0a4-95be21527653/machine-api-operator/ 2026-01-21 12:26:11.277243 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-api_machine-api-operator-5694c8668f-bmpxp_5a264bb6-3e63-4411-b0a4-95be21527653/machine-api-operator/0.log 2026-01-21 12:26:11.277260 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-crc_d1b160f5dda77d281dd8e69ec8d817f9/ 2026-01-21 12:26:11.277276 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-crc_d1b160f5dda77d281dd8e69ec8d817f9/kube-rbac-proxy-crio/ 2026-01-21 12:26:11.277293 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-crc_d1b160f5dda77d281dd8e69ec8d817f9/kube-rbac-proxy-crio/0.log 2026-01-21 12:26:11.277309 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-crc_d1b160f5dda77d281dd8e69ec8d817f9/setup/ 2026-01-21 12:26:11.277325 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_kube-rbac-proxy-crio-crc_d1b160f5dda77d281dd8e69ec8d817f9/setup/0.log 2026-01-21 12:26:11.277342 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-controller-84d6567774-2d2sj_6405bedd-bfe2-411b-937d-8f309fc6d0e8/ 2026-01-21 12:26:11.277358 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-controller-84d6567774-2d2sj_6405bedd-bfe2-411b-937d-8f309fc6d0e8/kube-rbac-proxy/ 2026-01-21 12:26:11.277374 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-controller-84d6567774-2d2sj_6405bedd-bfe2-411b-937d-8f309fc6d0e8/kube-rbac-proxy/0.log 2026-01-21 12:26:11.277390 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-controller-84d6567774-2d2sj_6405bedd-bfe2-411b-937d-8f309fc6d0e8/machine-config-controller/ 2026-01-21 12:26:11.277410 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-controller-84d6567774-2d2sj_6405bedd-bfe2-411b-937d-8f309fc6d0e8/machine-config-controller/0.log 2026-01-21 12:26:11.277427 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-daemon-rzs4q_f21c81eb-6979-46c3-9594-e4916d36fb0a/ 2026-01-21 12:26:11.277443 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-daemon-rzs4q_f21c81eb-6979-46c3-9594-e4916d36fb0a/kube-rbac-proxy/ 2026-01-21 12:26:11.277459 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-daemon-rzs4q_f21c81eb-6979-46c3-9594-e4916d36fb0a/kube-rbac-proxy/0.log 2026-01-21 12:26:11.277475 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-daemon-rzs4q_f21c81eb-6979-46c3-9594-e4916d36fb0a/machine-config-daemon/ 2026-01-21 12:26:11.277491 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-daemon-rzs4q_f21c81eb-6979-46c3-9594-e4916d36fb0a/machine-config-daemon/16.log 2026-01-21 12:26:11.277507 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-daemon-rzs4q_f21c81eb-6979-46c3-9594-e4916d36fb0a/machine-config-daemon/17.log 2026-01-21 12:26:11.277523 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-operator-74547568cd-q4cqt_91b59e6a-27bf-49a6-99c0-cb20160980ac/ 2026-01-21 12:26:11.277539 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-operator-74547568cd-q4cqt_91b59e6a-27bf-49a6-99c0-cb20160980ac/kube-rbac-proxy/ 2026-01-21 12:26:11.277555 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-operator-74547568cd-q4cqt_91b59e6a-27bf-49a6-99c0-cb20160980ac/kube-rbac-proxy/0.log 2026-01-21 12:26:11.277571 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-operator-74547568cd-q4cqt_91b59e6a-27bf-49a6-99c0-cb20160980ac/machine-config-operator/ 2026-01-21 12:26:11.277587 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-operator-74547568cd-q4cqt_91b59e6a-27bf-49a6-99c0-cb20160980ac/machine-config-operator/0.log 2026-01-21 12:26:11.277603 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-server-2hltc_10a74ffd-f45d-44ec-8287-d6a839fbb0af/ 2026-01-21 12:26:11.277619 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-server-2hltc_10a74ffd-f45d-44ec-8287-d6a839fbb0af/machine-config-server/ 2026-01-21 12:26:11.277634 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-machine-config-operator_machine-config-server-2hltc_10a74ffd-f45d-44ec-8287-d6a839fbb0af/machine-config-server/0.log 2026-01-21 12:26:11.277650 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/ 2026-01-21 12:26:11.277666 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/extract/ 2026-01-21 12:26:11.277700 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/extract/0.log 2026-01-21 12:26:11.277721 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/pull/ 2026-01-21 12:26:11.277738 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/pull/0.log 2026-01-21 12:26:11.277755 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/util/ 2026-01-21 12:26:11.277771 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_1f59f640c8a0eb1a7b0f26c81382bbdde784d03eb439a940bb8da3931arj2lm_b4b9e49f-2140-42de-b29f-6241bafc109e/util/0.log 2026-01-21 12:26:11.277787 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/ 2026-01-21 12:26:11.277803 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/extract/ 2026-01-21 12:26:11.277819 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/extract/0.log 2026-01-21 12:26:11.277835 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/pull/ 2026-01-21 12:26:11.277851 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/pull/0.log 2026-01-21 12:26:11.277867 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/util/ 2026-01-21 12:26:11.277883 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_270996307cd21d144be796860235064b5127c2fcf62ccccd6689c259dchvg76_5dfe3134-6d2f-47d9-b786-f69a4fbcf164/util/0.log 2026-01-21 12:26:11.277899 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/ 2026-01-21 12:26:11.277915 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/extract/ 2026-01-21 12:26:11.277931 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/extract/0.log 2026-01-21 12:26:11.277948 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/pull/ 2026-01-21 12:26:11.277964 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/pull/0.log 2026-01-21 12:26:11.277981 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/util/ 2026-01-21 12:26:11.278001 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_53efe8611d43ac2275911d954e05efbbba7920a530aff9253ed1cec713jdzzg_e058b309-ed1b-4162-a8e1-adf175ab47cf/util/0.log 2026-01-21 12:26:11.278017 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/ 2026-01-21 12:26:11.278034 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/extract/ 2026-01-21 12:26:11.278051 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/extract/0.log 2026-01-21 12:26:11.278068 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/pull/ 2026-01-21 12:26:11.278084 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/pull/0.log 2026-01-21 12:26:11.278101 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/util/ 2026-01-21 12:26:11.278117 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_98629960b44b381d1a86cff1d1439a8df43509c9ad24579158c59d0f08nj8kh_b8412bf3-79ed-4401-a927-e30a8a770afc/util/0.log 2026-01-21 12:26:11.278134 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/ 2026-01-21 12:26:11.278150 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/extract-content/ 2026-01-21 12:26:11.278167 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/extract-content/0.log 2026-01-21 12:26:11.278183 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/extract-utilities/ 2026-01-21 12:26:11.278199 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/extract-utilities/0.log 2026-01-21 12:26:11.278215 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/registry-server/ 2026-01-21 12:26:11.278232 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_certified-operators-xwrpb_c14bf9ba-8bf3-4a6d-a15b-0f62bc86d426/registry-server/0.log.gz 2026-01-21 12:26:11.278249 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/ 2026-01-21 12:26:11.278265 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/extract-content/ 2026-01-21 12:26:11.278282 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/extract-content/0.log 2026-01-21 12:26:11.278298 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/extract-utilities/ 2026-01-21 12:26:11.278318 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/extract-utilities/0.log 2026-01-21 12:26:11.278334 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/registry-server/ 2026-01-21 12:26:11.278350 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_community-operators-xbt62_19b1508e-f4b4-420f-abc7-d2c922cea0fc/registry-server/0.log.gz 2026-01-21 12:26:11.278366 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_marketplace-operator-79b997595-kzv24_9821d9aa-a481-43fd-a938-98d978d17299/ 2026-01-21 12:26:11.278382 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_marketplace-operator-79b997595-kzv24_9821d9aa-a481-43fd-a938-98d978d17299/marketplace-operator/ 2026-01-21 12:26:11.278398 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_marketplace-operator-79b997595-kzv24_9821d9aa-a481-43fd-a938-98d978d17299/marketplace-operator/0.log 2026-01-21 12:26:11.278414 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/ 2026-01-21 12:26:11.278430 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/extract-content/ 2026-01-21 12:26:11.278446 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/extract-content/0.log 2026-01-21 12:26:11.278462 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/extract-utilities/ 2026-01-21 12:26:11.278478 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/extract-utilities/0.log 2026-01-21 12:26:11.278495 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/registry-server/ 2026-01-21 12:26:11.278510 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-marketplace-bbvfx_d658610f-6e84-446d-9d81-e4e4198a6102/registry-server/0.log.gz 2026-01-21 12:26:11.278526 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/ 2026-01-21 12:26:11.278542 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/extract-content/ 2026-01-21 12:26:11.278558 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/extract-content/0.log 2026-01-21 12:26:11.278575 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/extract-utilities/ 2026-01-21 12:26:11.278591 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/extract-utilities/0.log 2026-01-21 12:26:11.278608 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/registry-server/ 2026-01-21 12:26:11.278625 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-marketplace_redhat-operators-mldwz_c49d0579-4622-43ef-a28d-7cbf66ce5998/registry-server/0.log.gz 2026-01-21 12:26:11.278644 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/ 2026-01-21 12:26:11.278715 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/bond-cni-plugin/ 2026-01-21 12:26:11.278737 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/bond-cni-plugin/0.log 2026-01-21 12:26:11.278754 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/cni-plugins/ 2026-01-21 12:26:11.278771 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/cni-plugins/0.log 2026-01-21 12:26:11.278788 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/egress-router-binary-copy/ 2026-01-21 12:26:11.278804 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/egress-router-binary-copy/0.log 2026-01-21 12:26:11.278820 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/kube-multus-additional-cni-plugins/ 2026-01-21 12:26:11.278836 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/kube-multus-additional-cni-plugins/0.log 2026-01-21 12:26:11.278852 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/routeoverride-cni/ 2026-01-21 12:26:11.278868 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/routeoverride-cni/0.log 2026-01-21 12:26:11.278884 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/whereabouts-cni-bincopy/ 2026-01-21 12:26:11.278901 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/whereabouts-cni-bincopy/0.log 2026-01-21 12:26:11.278916 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/whereabouts-cni/ 2026-01-21 12:26:11.278932 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-additional-cni-plugins-pbw2x_2b0b25f1-8430-459d-9805-e667615dc073/whereabouts-cni/0.log 2026-01-21 12:26:11.278948 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-admission-controller-857f4d67dd-k9srb_bebe6bc4-7b86-4688-ab28-408d5fc1ed7e/ 2026-01-21 12:26:11.278964 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-admission-controller-857f4d67dd-k9srb_bebe6bc4-7b86-4688-ab28-408d5fc1ed7e/kube-rbac-proxy/ 2026-01-21 12:26:11.278981 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-admission-controller-857f4d67dd-k9srb_bebe6bc4-7b86-4688-ab28-408d5fc1ed7e/kube-rbac-proxy/0.log 2026-01-21 12:26:11.278996 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-admission-controller-857f4d67dd-k9srb_bebe6bc4-7b86-4688-ab28-408d5fc1ed7e/multus-admission-controller/ 2026-01-21 12:26:11.279016 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-admission-controller-857f4d67dd-k9srb_bebe6bc4-7b86-4688-ab28-408d5fc1ed7e/multus-admission-controller/0.log 2026-01-21 12:26:11.279033 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-hwzqb_82b678c3-b1e1-4294-9f9f-02103a6823cc/ 2026-01-21 12:26:11.279048 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-hwzqb_82b678c3-b1e1-4294-9f9f-02103a6823cc/kube-multus/ 2026-01-21 12:26:11.279065 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-hwzqb_82b678c3-b1e1-4294-9f9f-02103a6823cc/kube-multus/2.log 2026-01-21 12:26:11.279080 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_multus-hwzqb_82b678c3-b1e1-4294-9f9f-02103a6823cc/kube-multus/3.log 2026-01-21 12:26:11.279096 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_network-metrics-daemon-2txwq_5c3596d1-1f08-4703-ab63-c29358aac0d9/ 2026-01-21 12:26:11.279112 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_network-metrics-daemon-2txwq_5c3596d1-1f08-4703-ab63-c29358aac0d9/kube-rbac-proxy/ 2026-01-21 12:26:11.279128 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_network-metrics-daemon-2txwq_5c3596d1-1f08-4703-ab63-c29358aac0d9/kube-rbac-proxy/0.log 2026-01-21 12:26:11.279144 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_network-metrics-daemon-2txwq_5c3596d1-1f08-4703-ab63-c29358aac0d9/network-metrics-daemon/ 2026-01-21 12:26:11.279160 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-multus_network-metrics-daemon-2txwq_5c3596d1-1f08-4703-ab63-c29358aac0d9/network-metrics-daemon/0.log 2026-01-21 12:26:11.279175 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-must-gather-tws8s_must-gather-q7jxq_4f4d2099-5115-4065-971b-e51c63e6ee05/ 2026-01-21 12:26:11.279191 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-must-gather-tws8s_must-gather-q7jxq_4f4d2099-5115-4065-971b-e51c63e6ee05/copy/ 2026-01-21 12:26:11.279207 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-must-gather-tws8s_must-gather-q7jxq_4f4d2099-5115-4065-971b-e51c63e6ee05/copy/0.log 2026-01-21 12:26:11.279224 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-must-gather-tws8s_must-gather-q7jxq_4f4d2099-5115-4065-971b-e51c63e6ee05/gather/ 2026-01-21 12:26:11.279241 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-must-gather-tws8s_must-gather-q7jxq_4f4d2099-5115-4065-971b-e51c63e6ee05/gather/0.log 2026-01-21 12:26:11.279257 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-console_networking-console-plugin-85b44fc459-gdk6g_5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/ 2026-01-21 12:26:11.279274 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-console_networking-console-plugin-85b44fc459-gdk6g_5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/networking-console-plugin/ 2026-01-21 12:26:11.279291 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-console_networking-console-plugin-85b44fc459-gdk6g_5fe485a1-e14f-4c09-b5b9-f252bc42b7e8/networking-console-plugin/0.log 2026-01-21 12:26:11.279307 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-diagnostics_network-check-source-55646444c4-trplf_9d751cbb-f2e2-430d-9754-c882a5e924a5/ 2026-01-21 12:26:11.279331 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-diagnostics_network-check-source-55646444c4-trplf_9d751cbb-f2e2-430d-9754-c882a5e924a5/check-endpoints/ 2026-01-21 12:26:11.279349 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-diagnostics_network-check-source-55646444c4-trplf_9d751cbb-f2e2-430d-9754-c882a5e924a5/check-endpoints/0.log 2026-01-21 12:26:11.279366 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-diagnostics_network-check-target-xd92c_3b6479f0-333b-4a96-9adf-2099afdc2447/ 2026-01-21 12:26:11.279382 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-diagnostics_network-check-target-xd92c_3b6479f0-333b-4a96-9adf-2099afdc2447/network-check-target-container/ 2026-01-21 12:26:11.279399 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-diagnostics_network-check-target-xd92c_3b6479f0-333b-4a96-9adf-2099afdc2447/network-check-target-container/0.log 2026-01-21 12:26:11.279416 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-node-identity_network-node-identity-vrzqb_ef543e1b-8068-4ea3-b32a-61027b32e95d/ 2026-01-21 12:26:11.279434 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-node-identity_network-node-identity-vrzqb_ef543e1b-8068-4ea3-b32a-61027b32e95d/approver/ 2026-01-21 12:26:11.279451 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-node-identity_network-node-identity-vrzqb_ef543e1b-8068-4ea3-b32a-61027b32e95d/approver/0.log 2026-01-21 12:26:11.279467 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-node-identity_network-node-identity-vrzqb_ef543e1b-8068-4ea3-b32a-61027b32e95d/webhook/ 2026-01-21 12:26:11.279483 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-node-identity_network-node-identity-vrzqb_ef543e1b-8068-4ea3-b32a-61027b32e95d/webhook/0.log 2026-01-21 12:26:11.279500 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-operator_iptables-alerter-4ln5h_d75a4c96-2883-4a0b-bab2-0fab2b6c0b49/ 2026-01-21 12:26:11.279517 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-operator_iptables-alerter-4ln5h_d75a4c96-2883-4a0b-bab2-0fab2b6c0b49/iptables-alerter/ 2026-01-21 12:26:11.279533 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-operator_iptables-alerter-4ln5h_d75a4c96-2883-4a0b-bab2-0fab2b6c0b49/iptables-alerter/0.log 2026-01-21 12:26:11.279550 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-operator_network-operator-58b4c7f79c-55gtf_37a5e44f-9a88-4405-be8a-b645485e7312/ 2026-01-21 12:26:11.279566 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-operator_network-operator-58b4c7f79c-55gtf_37a5e44f-9a88-4405-be8a-b645485e7312/network-operator/ 2026-01-21 12:26:11.279582 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-network-operator_network-operator-58b4c7f79c-55gtf_37a5e44f-9a88-4405-be8a-b645485e7312/network-operator/0.log 2026-01-21 12:26:11.279597 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-console-plugin-7754f76f8b-58tbd_eac14392-e11f-4bf9-b1db-d6200c0d0821/ 2026-01-21 12:26:11.279613 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-console-plugin-7754f76f8b-58tbd_eac14392-e11f-4bf9-b1db-d6200c0d0821/nmstate-console-plugin/ 2026-01-21 12:26:11.279629 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-console-plugin-7754f76f8b-58tbd_eac14392-e11f-4bf9-b1db-d6200c0d0821/nmstate-console-plugin/0.log 2026-01-21 12:26:11.279649 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-handler-j7spk_a3532c91-ec04-4a0c-99d3-bf6bf96a8887/ 2026-01-21 12:26:11.279666 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-handler-j7spk_a3532c91-ec04-4a0c-99d3-bf6bf96a8887/nmstate-handler/ 2026-01-21 12:26:11.279701 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-handler-j7spk_a3532c91-ec04-4a0c-99d3-bf6bf96a8887/nmstate-handler/0.log 2026-01-21 12:26:11.279719 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-metrics-54757c584b-c5llf_e125e9a3-31e1-47ce-ab99-c65ace2a60ec/ 2026-01-21 12:26:11.279736 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-metrics-54757c584b-c5llf_e125e9a3-31e1-47ce-ab99-c65ace2a60ec/kube-rbac-proxy/ 2026-01-21 12:26:11.279753 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-metrics-54757c584b-c5llf_e125e9a3-31e1-47ce-ab99-c65ace2a60ec/kube-rbac-proxy/0.log 2026-01-21 12:26:11.279769 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-metrics-54757c584b-c5llf_e125e9a3-31e1-47ce-ab99-c65ace2a60ec/nmstate-metrics/ 2026-01-21 12:26:11.279787 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-metrics-54757c584b-c5llf_e125e9a3-31e1-47ce-ab99-c65ace2a60ec/nmstate-metrics/0.log 2026-01-21 12:26:11.279805 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-operator-646758c888-8kzgs_88c3cb65-2aff-44db-85fb-8c365c93439f/ 2026-01-21 12:26:11.279822 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-operator-646758c888-8kzgs_88c3cb65-2aff-44db-85fb-8c365c93439f/nmstate-operator/ 2026-01-21 12:26:11.279837 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-operator-646758c888-8kzgs_88c3cb65-2aff-44db-85fb-8c365c93439f/nmstate-operator/0.log 2026-01-21 12:26:11.279853 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-webhook-8474b5b9d8-87q5g_3c739657-6960-46fe-be71-6d965b98e714/ 2026-01-21 12:26:11.279869 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-webhook-8474b5b9d8-87q5g_3c739657-6960-46fe-be71-6d965b98e714/nmstate-webhook/ 2026-01-21 12:26:11.279885 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-nmstate_nmstate-webhook-8474b5b9d8-87q5g_3c739657-6960-46fe-be71-6d965b98e714/nmstate-webhook/0.log 2026-01-21 12:26:11.279900 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-oauth-apiserver_apiserver-7bbb656c7d-dnjzx_5ce9d1cf-7d42-4a6a-91ce-3a1d0afd4605/ 2026-01-21 12:26:11.279915 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-oauth-apiserver_apiserver-7bbb656c7d-dnjzx_5ce9d1cf-7d42-4a6a-91ce-3a1d0afd4605/fix-audit-permissions/ 2026-01-21 12:26:11.279930 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-oauth-apiserver_apiserver-7bbb656c7d-dnjzx_5ce9d1cf-7d42-4a6a-91ce-3a1d0afd4605/fix-audit-permissions/0.log 2026-01-21 12:26:11.279945 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-oauth-apiserver_apiserver-7bbb656c7d-dnjzx_5ce9d1cf-7d42-4a6a-91ce-3a1d0afd4605/oauth-apiserver/ 2026-01-21 12:26:11.279960 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-oauth-apiserver_apiserver-7bbb656c7d-dnjzx_5ce9d1cf-7d42-4a6a-91ce-3a1d0afd4605/oauth-apiserver/0.log 2026-01-21 12:26:11.279975 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_catalog-operator-68c6474976-krz9k_b22e36ec-37f7-4d2c-87fb-ce56d5436a8d/ 2026-01-21 12:26:11.279993 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_catalog-operator-68c6474976-krz9k_b22e36ec-37f7-4d2c-87fb-ce56d5436a8d/catalog-operator/ 2026-01-21 12:26:11.280009 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_catalog-operator-68c6474976-krz9k_b22e36ec-37f7-4d2c-87fb-ce56d5436a8d/catalog-operator/0.log.gz 2026-01-21 12:26:11.280024 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483265-mgz4z_3325e422-dbe5-4937-a209-f3a477899616/ 2026-01-21 12:26:11.280054 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483265-mgz4z_3325e422-dbe5-4937-a209-f3a477899616/collect-profiles/ 2026-01-21 12:26:11.280071 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483265-mgz4z_3325e422-dbe5-4937-a209-f3a477899616/collect-profiles/0.log 2026-01-21 12:26:11.280087 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483280-2nvt2_b7b5dd29-371b-47b9-8b77-a0395637b4d9/ 2026-01-21 12:26:11.280103 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483280-2nvt2_b7b5dd29-371b-47b9-8b77-a0395637b4d9/collect-profiles/ 2026-01-21 12:26:11.280119 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483280-2nvt2_b7b5dd29-371b-47b9-8b77-a0395637b4d9/collect-profiles/0.log 2026-01-21 12:26:11.280135 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483295-xjvfh_ad9fb0e6-2a5f-4699-8a43-bb7cc8bc05ab/ 2026-01-21 12:26:11.280152 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483295-xjvfh_ad9fb0e6-2a5f-4699-8a43-bb7cc8bc05ab/collect-profiles/ 2026-01-21 12:26:11.280168 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_collect-profiles-29483295-xjvfh_ad9fb0e6-2a5f-4699-8a43-bb7cc8bc05ab/collect-profiles/0.log 2026-01-21 12:26:11.280184 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_olm-operator-6b444d44fb-z9nt4_c98e3838-5bee-44ee-8fca-b5b429cef61e/ 2026-01-21 12:26:11.280199 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_olm-operator-6b444d44fb-z9nt4_c98e3838-5bee-44ee-8fca-b5b429cef61e/olm-operator/ 2026-01-21 12:26:11.280215 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_olm-operator-6b444d44fb-z9nt4_c98e3838-5bee-44ee-8fca-b5b429cef61e/olm-operator/0.log 2026-01-21 12:26:11.280231 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_package-server-manager-789f6589d5-fvwpn_4d61279b-b5c2-440f-9bac-689a27484f8c/ 2026-01-21 12:26:11.280247 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_package-server-manager-789f6589d5-fvwpn_4d61279b-b5c2-440f-9bac-689a27484f8c/kube-rbac-proxy/ 2026-01-21 12:26:11.280262 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_package-server-manager-789f6589d5-fvwpn_4d61279b-b5c2-440f-9bac-689a27484f8c/kube-rbac-proxy/0.log 2026-01-21 12:26:11.280278 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_package-server-manager-789f6589d5-fvwpn_4d61279b-b5c2-440f-9bac-689a27484f8c/package-server-manager/ 2026-01-21 12:26:11.280298 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_package-server-manager-789f6589d5-fvwpn_4d61279b-b5c2-440f-9bac-689a27484f8c/package-server-manager/0.log 2026-01-21 12:26:11.280315 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_packageserver-d55dfcdfc-5l4bq_be085cb8-85ab-409e-a8cb-3d02cd7153f6/ 2026-01-21 12:26:11.280332 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_packageserver-d55dfcdfc-5l4bq_be085cb8-85ab-409e-a8cb-3d02cd7153f6/packageserver/ 2026-01-21 12:26:11.280348 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operator-lifecycle-manager_packageserver-d55dfcdfc-5l4bq_be085cb8-85ab-409e-a8cb-3d02cd7153f6/packageserver/0.log 2026-01-21 12:26:11.280365 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-68bc856cb9-x48ml_5331ad9e-1914-414a-a7b2-b52eb191ba2f/ 2026-01-21 12:26:11.280382 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-68bc856cb9-x48ml_5331ad9e-1914-414a-a7b2-b52eb191ba2f/prometheus-operator/ 2026-01-21 12:26:11.280398 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-68bc856cb9-x48ml_5331ad9e-1914-414a-a7b2-b52eb191ba2f/prometheus-operator/0.log 2026-01-21 12:26:11.280415 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-admission-webhook-58676c4f98-fgbpf_28aa0136-6b61-4a88-907d-265c48e36f08/ 2026-01-21 12:26:11.280431 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-admission-webhook-58676c4f98-fgbpf_28aa0136-6b61-4a88-907d-265c48e36f08/prometheus-operator-admission-webhook/ 2026-01-21 12:26:11.280448 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-admission-webhook-58676c4f98-fgbpf_28aa0136-6b61-4a88-907d-265c48e36f08/prometheus-operator-admission-webhook/0.log 2026-01-21 12:26:11.280465 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-admission-webhook-58676c4f98-l96bw_ea8b2f0b-f77a-4737-be37-3268437871d9/ 2026-01-21 12:26:11.280482 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-admission-webhook-58676c4f98-l96bw_ea8b2f0b-f77a-4737-be37-3268437871d9/prometheus-operator-admission-webhook/ 2026-01-21 12:26:11.280499 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_obo-prometheus-operator-admission-webhook-58676c4f98-l96bw_ea8b2f0b-f77a-4737-be37-3268437871d9/prometheus-operator-admission-webhook/0.log 2026-01-21 12:26:11.280516 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_observability-operator-59bdc8b94-jvt9p_e052dc8b-2520-4757-bb0a-d1350ad44b08/ 2026-01-21 12:26:11.280533 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_observability-operator-59bdc8b94-jvt9p_e052dc8b-2520-4757-bb0a-d1350ad44b08/operator/ 2026-01-21 12:26:11.280550 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_observability-operator-59bdc8b94-jvt9p_e052dc8b-2520-4757-bb0a-d1350ad44b08/operator/0.log 2026-01-21 12:26:11.280568 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_observability-ui-dashboards-66cbf594b5-mmj2s_a5dc6045-7192-42dc-b653-a71b80a9f119/ 2026-01-21 12:26:11.280585 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_observability-ui-dashboards-66cbf594b5-mmj2s_a5dc6045-7192-42dc-b653-a71b80a9f119/observability-ui-dashboards/ 2026-01-21 12:26:11.280606 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_observability-ui-dashboards-66cbf594b5-mmj2s_a5dc6045-7192-42dc-b653-a71b80a9f119/observability-ui-dashboards/0.log 2026-01-21 12:26:11.280624 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_perses-operator-5bf474d74f-655nk_0d031a33-73a8-45d7-9979-e1266d9e7be7/ 2026-01-21 12:26:11.280640 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_perses-operator-5bf474d74f-655nk_0d031a33-73a8-45d7-9979-e1266d9e7be7/perses-operator/ 2026-01-21 12:26:11.280657 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-operators_perses-operator-5bf474d74f-655nk_0d031a33-73a8-45d7-9979-e1266d9e7be7/perses-operator/0.log 2026-01-21 12:26:11.280674 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-control-plane-749d76644c-glmql_a8599a6b-48cb-400d-ac34-86be75b9ce54/ 2026-01-21 12:26:11.280709 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-control-plane-749d76644c-glmql_a8599a6b-48cb-400d-ac34-86be75b9ce54/kube-rbac-proxy/ 2026-01-21 12:26:11.280726 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-control-plane-749d76644c-glmql_a8599a6b-48cb-400d-ac34-86be75b9ce54/kube-rbac-proxy/0.log 2026-01-21 12:26:11.280743 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-control-plane-749d76644c-glmql_a8599a6b-48cb-400d-ac34-86be75b9ce54/ovnkube-cluster-manager/ 2026-01-21 12:26:11.280759 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-control-plane-749d76644c-glmql_a8599a6b-48cb-400d-ac34-86be75b9ce54/ovnkube-cluster-manager/0.log 2026-01-21 12:26:11.280776 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ 2026-01-21 12:26:11.280792 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/kube-rbac-proxy-node/ 2026-01-21 12:26:11.280808 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/kube-rbac-proxy-node/0.log 2026-01-21 12:26:11.280824 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/kube-rbac-proxy-ovn-metrics/ 2026-01-21 12:26:11.280840 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/kube-rbac-proxy-ovn-metrics/0.log 2026-01-21 12:26:11.280857 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/kubecfg-setup/ 2026-01-21 12:26:11.280873 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/kubecfg-setup/0.log 2026-01-21 12:26:11.280889 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/nbdb/ 2026-01-21 12:26:11.280905 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/nbdb/0.log 2026-01-21 12:26:11.280922 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/northd/ 2026-01-21 12:26:11.280941 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/northd/0.log 2026-01-21 12:26:11.280958 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ovn-acl-logging/ 2026-01-21 12:26:11.280975 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ovn-acl-logging/0.log 2026-01-21 12:26:11.280992 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ovn-controller/ 2026-01-21 12:26:11.281008 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ovn-controller/0.log 2026-01-21 12:26:11.281024 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ovnkube-controller/ 2026-01-21 12:26:11.281040 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/ovnkube-controller/0.log.gz 2026-01-21 12:26:11.281056 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/sbdb/ 2026-01-21 12:26:11.281071 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-ovn-kubernetes_ovnkube-node-pfrnv_0ca4638f-b097-4f42-9a44-c429d8c3e6b2/sbdb/0.log 2026-01-21 12:26:11.281087 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-route-controller-manager_route-controller-manager-654bb78b8-7zdlr_72af1efd-5fd6-4bc9-9e3b-eb85947c088f/ 2026-01-21 12:26:11.281103 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-route-controller-manager_route-controller-manager-654bb78b8-7zdlr_72af1efd-5fd6-4bc9-9e3b-eb85947c088f/route-controller-manager/ 2026-01-21 12:26:11.281120 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-route-controller-manager_route-controller-manager-654bb78b8-7zdlr_72af1efd-5fd6-4bc9-9e3b-eb85947c088f/route-controller-manager/0.log 2026-01-21 12:26:11.281136 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-service-ca-operator_service-ca-operator-777779d784-ntfc2_e265c5ca-d8d4-4ba0-81db-fd48d3974762/ 2026-01-21 12:26:11.281151 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-service-ca-operator_service-ca-operator-777779d784-ntfc2_e265c5ca-d8d4-4ba0-81db-fd48d3974762/service-ca-operator/ 2026-01-21 12:26:11.281167 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-service-ca-operator_service-ca-operator-777779d784-ntfc2_e265c5ca-d8d4-4ba0-81db-fd48d3974762/service-ca-operator/0.log 2026-01-21 12:26:11.281183 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-service-ca_service-ca-9c57cc56f-bhfrw_434007d9-38af-49cd-a16f-09c87531b8c1/ 2026-01-21 12:26:11.281199 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-service-ca_service-ca-9c57cc56f-bhfrw_434007d9-38af-49cd-a16f-09c87531b8c1/service-ca-controller/ 2026-01-21 12:26:11.281216 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openshift-service-ca_service-ca-9c57cc56f-bhfrw_434007d9-38af-49cd-a16f-09c87531b8c1/service-ca-controller/0.log 2026-01-21 12:26:11.281232 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/ 2026-01-21 12:26:11.281252 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/extract/ 2026-01-21 12:26:11.281268 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/extract/0.log 2026-01-21 12:26:11.281284 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/pull/ 2026-01-21 12:26:11.281300 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/pull/0.log 2026-01-21 12:26:11.281317 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/util/ 2026-01-21 12:26:11.281495 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_144d34eb9a01f64553f418551f09aae5c8acbbe5dabd7b229fc5e5452fms2cx_9c5f0822-d0f9-4273-8e7f-e2f91d277a01/util/0.log 2026-01-21 12:26:11.281525 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/ 2026-01-21 12:26:11.281543 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/extract/ 2026-01-21 12:26:11.281560 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/extract/0.log 2026-01-21 12:26:11.281577 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/pull/ 2026-01-21 12:26:11.281593 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/pull/0.log 2026-01-21 12:26:11.281608 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/util/ 2026-01-21 12:26:11.281625 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_23550c8618544ac9ea89afd4ce99cda9256ff69faea7c95bed8068d414hqt75_06c4add2-f00d-4aea-8168-f165cdf2b7cf/util/0.log 2026-01-21 12:26:11.281642 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_barbican-operator-controller-manager-7ddb5c749-r4klh_d8031329-a6ad-49da-881e-94db9f545ab7/ 2026-01-21 12:26:11.281659 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_barbican-operator-controller-manager-7ddb5c749-r4klh_d8031329-a6ad-49da-881e-94db9f545ab7/manager/ 2026-01-21 12:26:11.281689 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_barbican-operator-controller-manager-7ddb5c749-r4klh_d8031329-a6ad-49da-881e-94db9f545ab7/manager/0.log 2026-01-21 12:26:11.281712 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_cinder-operator-controller-manager-9b68f5989-vmg65_cf77bf31-5d25-4015-b274-05dbedbedf5a/ 2026-01-21 12:26:11.281732 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_cinder-operator-controller-manager-9b68f5989-vmg65_cf77bf31-5d25-4015-b274-05dbedbedf5a/manager/ 2026-01-21 12:26:11.281753 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_cinder-operator-controller-manager-9b68f5989-vmg65_cf77bf31-5d25-4015-b274-05dbedbedf5a/manager/0.log 2026-01-21 12:26:11.281771 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_designate-operator-controller-manager-9f958b845-mjg4d_50c322c0-a941-48fa-bf86-c2daa64a9aa8/ 2026-01-21 12:26:11.281789 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_designate-operator-controller-manager-9f958b845-mjg4d_50c322c0-a941-48fa-bf86-c2daa64a9aa8/manager/ 2026-01-21 12:26:11.281805 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_designate-operator-controller-manager-9f958b845-mjg4d_50c322c0-a941-48fa-bf86-c2daa64a9aa8/manager/0.log 2026-01-21 12:26:11.281822 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_glance-operator-controller-manager-c6994669c-ggpw9_e1fe1f25-b3e4-4ed9-a44b-e9bdf6eedd23/ 2026-01-21 12:26:11.281839 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_glance-operator-controller-manager-c6994669c-ggpw9_e1fe1f25-b3e4-4ed9-a44b-e9bdf6eedd23/manager/ 2026-01-21 12:26:11.281855 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_glance-operator-controller-manager-c6994669c-ggpw9_e1fe1f25-b3e4-4ed9-a44b-e9bdf6eedd23/manager/0.log 2026-01-21 12:26:11.281872 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_heat-operator-controller-manager-594c8c9d5d-h9szq_9b9f5cfa-93e1-4940-b7f0-066c6bc4f194/ 2026-01-21 12:26:11.281888 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_heat-operator-controller-manager-594c8c9d5d-h9szq_9b9f5cfa-93e1-4940-b7f0-066c6bc4f194/manager/ 2026-01-21 12:26:11.281905 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_heat-operator-controller-manager-594c8c9d5d-h9szq_9b9f5cfa-93e1-4940-b7f0-066c6bc4f194/manager/0.log 2026-01-21 12:26:11.281922 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_horizon-operator-controller-manager-77d5c5b54f-gcxp4_c4be49a0-e872-456f-a102-928f5210524f/ 2026-01-21 12:26:11.281939 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_horizon-operator-controller-manager-77d5c5b54f-gcxp4_c4be49a0-e872-456f-a102-928f5210524f/manager/ 2026-01-21 12:26:11.281955 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_horizon-operator-controller-manager-77d5c5b54f-gcxp4_c4be49a0-e872-456f-a102-928f5210524f/manager/0.log 2026-01-21 12:26:11.281972 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_infra-operator-controller-manager-77c48c7859-dqjpf_dbe9a043-a969-429b-b7b1-33d12296c52c/ 2026-01-21 12:26:11.281989 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_infra-operator-controller-manager-77c48c7859-dqjpf_dbe9a043-a969-429b-b7b1-33d12296c52c/manager/ 2026-01-21 12:26:11.282006 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_infra-operator-controller-manager-77c48c7859-dqjpf_dbe9a043-a969-429b-b7b1-33d12296c52c/manager/0.log 2026-01-21 12:26:11.282168 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_ironic-operator-controller-manager-78757b4889-wdwvl_2d8c2e69-7444-465a-a418-59d9c5b20074/ 2026-01-21 12:26:11.282190 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_ironic-operator-controller-manager-78757b4889-wdwvl_2d8c2e69-7444-465a-a418-59d9c5b20074/manager/ 2026-01-21 12:26:11.282208 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_ironic-operator-controller-manager-78757b4889-wdwvl_2d8c2e69-7444-465a-a418-59d9c5b20074/manager/0.log 2026-01-21 12:26:11.282230 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_keystone-operator-controller-manager-767fdc4f47-2znsh_2c47ce4c-9012-4798-9bf8-127a96ad285e/ 2026-01-21 12:26:11.282248 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_keystone-operator-controller-manager-767fdc4f47-2znsh_2c47ce4c-9012-4798-9bf8-127a96ad285e/manager/ 2026-01-21 12:26:11.282264 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_keystone-operator-controller-manager-767fdc4f47-2znsh_2c47ce4c-9012-4798-9bf8-127a96ad285e/manager/0.log 2026-01-21 12:26:11.282281 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_manila-operator-controller-manager-864f6b75bf-x7474_fc8ec38e-f941-4ba0-863e-933e10bf2043/ 2026-01-21 12:26:11.282440 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_manila-operator-controller-manager-864f6b75bf-x7474_fc8ec38e-f941-4ba0-863e-933e10bf2043/manager/ 2026-01-21 12:26:11.282751 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_manila-operator-controller-manager-864f6b75bf-x7474_fc8ec38e-f941-4ba0-863e-933e10bf2043/manager/0.log 2026-01-21 12:26:11.282777 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_mariadb-operator-controller-manager-c87fff755-b4cd2_6d27cfd1-683a-4e92-bcaf-40f1f370cd1b/ 2026-01-21 12:26:11.282796 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_mariadb-operator-controller-manager-c87fff755-b4cd2_6d27cfd1-683a-4e92-bcaf-40f1f370cd1b/manager/ 2026-01-21 12:26:11.282813 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_mariadb-operator-controller-manager-c87fff755-b4cd2_6d27cfd1-683a-4e92-bcaf-40f1f370cd1b/manager/0.log 2026-01-21 12:26:11.282831 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_neutron-operator-controller-manager-cb4666565-t9fng_398ea514-c4f3-40db-8421-ebf007fda30d/ 2026-01-21 12:26:11.282847 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_neutron-operator-controller-manager-cb4666565-t9fng_398ea514-c4f3-40db-8421-ebf007fda30d/manager/ 2026-01-21 12:26:11.282863 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_neutron-operator-controller-manager-cb4666565-t9fng_398ea514-c4f3-40db-8421-ebf007fda30d/manager/0.log 2026-01-21 12:26:11.282879 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_nova-operator-controller-manager-65849867d6-nqldj_a7dd34dc-8a69-4c91-88ec-d1d7beffb15d/ 2026-01-21 12:26:11.282895 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_nova-operator-controller-manager-65849867d6-nqldj_a7dd34dc-8a69-4c91-88ec-d1d7beffb15d/manager/ 2026-01-21 12:26:11.282911 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_nova-operator-controller-manager-65849867d6-nqldj_a7dd34dc-8a69-4c91-88ec-d1d7beffb15d/manager/0.log 2026-01-21 12:26:11.282926 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_octavia-operator-controller-manager-7fc9b76cf6-44xwf_cc5d8922-f54d-42a1-b23a-622329e3f644/ 2026-01-21 12:26:11.282942 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_octavia-operator-controller-manager-7fc9b76cf6-44xwf_cc5d8922-f54d-42a1-b23a-622329e3f644/manager/ 2026-01-21 12:26:11.282958 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_octavia-operator-controller-manager-7fc9b76cf6-44xwf_cc5d8922-f54d-42a1-b23a-622329e3f644/manager/0.log 2026-01-21 12:26:11.282978 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-baremetal-operator-controller-manager-6b68b8b854hdtp9_05db7c08-87f6-4518-8d61-c87cbf0b1735/ 2026-01-21 12:26:11.282994 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-baremetal-operator-controller-manager-6b68b8b854hdtp9_05db7c08-87f6-4518-8d61-c87cbf0b1735/manager/ 2026-01-21 12:26:11.283010 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-baremetal-operator-controller-manager-6b68b8b854hdtp9_05db7c08-87f6-4518-8d61-c87cbf0b1735/manager/0.log 2026-01-21 12:26:11.283026 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-operator-controller-manager-87d6d564b-dgm28_be80c7ef-4f5f-4660-9954-5ab5b34655cf/ 2026-01-21 12:26:11.283042 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-operator-controller-manager-87d6d564b-dgm28_be80c7ef-4f5f-4660-9954-5ab5b34655cf/manager/ 2026-01-21 12:26:11.283057 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-operator-controller-manager-87d6d564b-dgm28_be80c7ef-4f5f-4660-9954-5ab5b34655cf/manager/0.log.gz 2026-01-21 12:26:11.283072 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-operator-index-6jmrz_25cc6d46-b21a-463f-a13a-9874780c87f3/ 2026-01-21 12:26:11.283217 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-operator-index-6jmrz_25cc6d46-b21a-463f-a13a-9874780c87f3/registry-server/ 2026-01-21 12:26:11.283244 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_openstack-operator-index-6jmrz_25cc6d46-b21a-463f-a13a-9874780c87f3/registry-server/0.log 2026-01-21 12:26:11.283263 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_ovn-operator-controller-manager-55db956ddc-l9f98_a032309d-2543-4e6b-8207-d8097dffcaf5/ 2026-01-21 12:26:11.283281 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_ovn-operator-controller-manager-55db956ddc-l9f98_a032309d-2543-4e6b-8207-d8097dffcaf5/manager/ 2026-01-21 12:26:11.283299 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_ovn-operator-controller-manager-55db956ddc-l9f98_a032309d-2543-4e6b-8207-d8097dffcaf5/manager/0.log 2026-01-21 12:26:11.283315 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_placement-operator-controller-manager-686df47fcb-84spn_0fb89ff9-2ba9-4a38-b739-43fa22a5b209/ 2026-01-21 12:26:11.283332 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_placement-operator-controller-manager-686df47fcb-84spn_0fb89ff9-2ba9-4a38-b739-43fa22a5b209/manager/ 2026-01-21 12:26:11.283348 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_placement-operator-controller-manager-686df47fcb-84spn_0fb89ff9-2ba9-4a38-b739-43fa22a5b209/manager/0.log 2026-01-21 12:26:11.283365 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_rabbitmq-cluster-operator-manager-668c99d594-k7r2f_182d9a34-f024-4a86-8851-9e20d654f4ac/ 2026-01-21 12:26:11.283381 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_rabbitmq-cluster-operator-manager-668c99d594-k7r2f_182d9a34-f024-4a86-8851-9e20d654f4ac/operator/ 2026-01-21 12:26:11.283397 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_rabbitmq-cluster-operator-manager-668c99d594-k7r2f_182d9a34-f024-4a86-8851-9e20d654f4ac/operator/0.log 2026-01-21 12:26:11.283413 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_swift-operator-controller-manager-85dd56d4cc-cq4k9_a9c52af6-912a-4e93-bbcd-42e961453471/ 2026-01-21 12:26:11.283433 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_swift-operator-controller-manager-85dd56d4cc-cq4k9_a9c52af6-912a-4e93-bbcd-42e961453471/manager/ 2026-01-21 12:26:11.283449 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_swift-operator-controller-manager-85dd56d4cc-cq4k9_a9c52af6-912a-4e93-bbcd-42e961453471/manager/0.log 2026-01-21 12:26:11.283465 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_telemetry-operator-controller-manager-5f8f495fcf-gcwbr_d7429a44-6eeb-419b-8193-29275baf4ad9/ 2026-01-21 12:26:11.283482 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_telemetry-operator-controller-manager-5f8f495fcf-gcwbr_d7429a44-6eeb-419b-8193-29275baf4ad9/manager/ 2026-01-21 12:26:11.283498 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_telemetry-operator-controller-manager-5f8f495fcf-gcwbr_d7429a44-6eeb-419b-8193-29275baf4ad9/manager/0.log.gz 2026-01-21 12:26:11.283514 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_test-operator-controller-manager-7cd8bc9dbb-hvtnz_fd15c43d-a647-467e-a4f1-eb0ca81a123f/ 2026-01-21 12:26:11.283530 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_test-operator-controller-manager-7cd8bc9dbb-hvtnz_fd15c43d-a647-467e-a4f1-eb0ca81a123f/manager/ 2026-01-21 12:26:11.283546 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_test-operator-controller-manager-7cd8bc9dbb-hvtnz_fd15c43d-a647-467e-a4f1-eb0ca81a123f/manager/0.log 2026-01-21 12:26:11.283563 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_watcher-operator-controller-manager-696df99475-8gncw_d8433174-98d9-44ec-924f-6fe639538b64/ 2026-01-21 12:26:11.283579 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_watcher-operator-controller-manager-696df99475-8gncw_d8433174-98d9-44ec-924f-6fe639538b64/manager/ 2026-01-21 12:26:11.283596 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_watcher-operator-controller-manager-696df99475-8gncw_d8433174-98d9-44ec-924f-6fe639538b64/manager/0.log.gz 2026-01-21 12:26:11.283612 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_watcher-operator-index-lvk6z_2ac527d2-36c7-40bc-ae76-8d007f3dadb3/ 2026-01-21 12:26:11.283629 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_watcher-operator-index-lvk6z_2ac527d2-36c7-40bc-ae76-8d007f3dadb3/registry-server/ 2026-01-21 12:26:11.283645 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/openstack-operators_watcher-operator-index-lvk6z_2ac527d2-36c7-40bc-ae76-8d007f3dadb3/registry-server/0.log 2026-01-21 12:26:11.283662 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/ 2026-01-21 12:26:11.283693 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/alertmanager/ 2026-01-21 12:26:11.283715 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/alertmanager/0.log 2026-01-21 12:26:11.283732 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/config-reloader/ 2026-01-21 12:26:11.283753 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/config-reloader/0.log 2026-01-21 12:26:11.283769 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/init-config-reloader/ 2026-01-21 12:26:11.283786 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_alertmanager-metric-storage-0_938ddfe0-198f-4050-af00-6c195ffaa41e/init-config-reloader/0.log 2026-01-21 12:26:11.283802 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/ 2026-01-21 12:26:11.283819 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/ceilometer-central-agent/ 2026-01-21 12:26:11.283835 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/ceilometer-central-agent/0.log 2026-01-21 12:26:11.283852 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/ceilometer-notification-agent/ 2026-01-21 12:26:11.283868 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/ceilometer-notification-agent/0.log 2026-01-21 12:26:11.283885 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/proxy-httpd/ 2026-01-21 12:26:11.283901 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/proxy-httpd/0.log 2026-01-21 12:26:11.283917 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/sg-core/ 2026-01-21 12:26:11.283933 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_ceilometer-0_eb3be111-53a6-456c-8c50-bf9b7c6cb367/sg-core/0.log 2026-01-21 12:26:11.283949 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_keystone-6cf7c7c58-dd2hr_86449d00-a2ae-4fb9-8529-e5a140d7b2f8/ 2026-01-21 12:26:11.283965 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_keystone-6cf7c7c58-dd2hr_86449d00-a2ae-4fb9-8529-e5a140d7b2f8/keystone-api/ 2026-01-21 12:26:11.283981 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_keystone-6cf7c7c58-dd2hr_86449d00-a2ae-4fb9-8529-e5a140d7b2f8/keystone-api/0.log 2026-01-21 12:26:11.283998 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_keystone-cron-29483281-2xgjf_a579ba5c-cb85-4070-a2ca-f2061d14e311/ 2026-01-21 12:26:11.284014 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_keystone-cron-29483281-2xgjf_a579ba5c-cb85-4070-a2ca-f2061d14e311/keystone-cron/ 2026-01-21 12:26:11.284030 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_keystone-cron-29483281-2xgjf_a579ba5c-cb85-4070-a2ca-f2061d14e311/keystone-cron/0.log 2026-01-21 12:26:11.284063 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_kube-state-metrics-0_36d577b6-db6f-4302-a839-ed148c56f7b6/ 2026-01-21 12:26:11.284080 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_kube-state-metrics-0_36d577b6-db6f-4302-a839-ed148c56f7b6/kube-state-metrics/ 2026-01-21 12:26:11.284097 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_kube-state-metrics-0_36d577b6-db6f-4302-a839-ed148c56f7b6/kube-state-metrics/0.log 2026-01-21 12:26:11.284117 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_memcached-0_1a2e82ce-ac09-4cf1-95cd-3c206a2a06ed/ 2026-01-21 12:26:11.284134 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_memcached-0_1a2e82ce-ac09-4cf1-95cd-3c206a2a06ed/memcached/ 2026-01-21 12:26:11.284151 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_memcached-0_1a2e82ce-ac09-4cf1-95cd-3c206a2a06ed/memcached/0.log.gz 2026-01-21 12:26:11.284167 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstack-galera-0_a76fd86c-08d3-47af-af39-e3336a2f5c0b/ 2026-01-21 12:26:11.284183 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstack-galera-0_a76fd86c-08d3-47af-af39-e3336a2f5c0b/galera/ 2026-01-21 12:26:11.284200 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstack-galera-0_a76fd86c-08d3-47af-af39-e3336a2f5c0b/galera/0.log 2026-01-21 12:26:11.284216 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstack-galera-0_a76fd86c-08d3-47af-af39-e3336a2f5c0b/mysql-bootstrap/ 2026-01-21 12:26:11.284232 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstack-galera-0_a76fd86c-08d3-47af-af39-e3336a2f5c0b/mysql-bootstrap/0.log 2026-01-21 12:26:11.284248 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstackclient_2c73a99a-7a7e-4746-9404-3dc64865ea05/ 2026-01-21 12:26:11.284265 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstackclient_2c73a99a-7a7e-4746-9404-3dc64865ea05/openstackclient/ 2026-01-21 12:26:11.284281 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_openstackclient_2c73a99a-7a7e-4746-9404-3dc64865ea05/openstackclient/0.log 2026-01-21 12:26:11.284297 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/ 2026-01-21 12:26:11.284313 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/config-reloader/ 2026-01-21 12:26:11.284329 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/config-reloader/0.log 2026-01-21 12:26:11.284345 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/init-config-reloader/ 2026-01-21 12:26:11.284361 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/init-config-reloader/0.log 2026-01-21 12:26:11.284378 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/prometheus/ 2026-01-21 12:26:11.284394 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/prometheus/0.log 2026-01-21 12:26:11.284410 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/thanos-sidecar/ 2026-01-21 12:26:11.284427 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_prometheus-metric-storage-0_74f59733-5086-4ebd-9e6a-764a947d38b4/thanos-sidecar/0.log 2026-01-21 12:26:11.284446 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-notifications-server-0_4c494924-513c-4575-a9c9-78e15c3751bc/ 2026-01-21 12:26:11.284463 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-notifications-server-0_4c494924-513c-4575-a9c9-78e15c3751bc/rabbitmq/ 2026-01-21 12:26:11.284479 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-notifications-server-0_4c494924-513c-4575-a9c9-78e15c3751bc/rabbitmq/0.log 2026-01-21 12:26:11.284495 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-notifications-server-0_4c494924-513c-4575-a9c9-78e15c3751bc/setup-container/ 2026-01-21 12:26:11.284510 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-notifications-server-0_4c494924-513c-4575-a9c9-78e15c3751bc/setup-container/0.log 2026-01-21 12:26:11.284526 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-server-0_b7c93089-4b7c-45c7-aa48-64622e536032/ 2026-01-21 12:26:11.284543 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-server-0_b7c93089-4b7c-45c7-aa48-64622e536032/rabbitmq/ 2026-01-21 12:26:11.284558 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-server-0_b7c93089-4b7c-45c7-aa48-64622e536032/rabbitmq/0.log 2026-01-21 12:26:11.284574 | controller | cd+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-server-0_b7c93089-4b7c-45c7-aa48-64622e536032/setup-container/ 2026-01-21 12:26:11.284709 | controller | >f+++++++++ ci-framework-data/logs/crc/crc-logs-artifacts/pods/watcher-kuttl-default_rabbitmq-server-0_b7c93089-4b7c-45c7-aa48-64622e536032/setup-container/0.log 2026-01-21 12:26:11.284739 | controller | cd+++++++++ ci-framework-data/logs/openstack-must-gather/ 2026-01-21 12:26:11.284760 | controller | >f+++++++++ ci-framework-data/logs/openstack-must-gather/must-gather.logs 2026-01-21 12:26:11.284779 | controller | >f+++++++++ ci-framework-data/logs/openstack-must-gather/timestamp 2026-01-21 12:26:11.284796 | controller | cd+++++++++ registries.conf.d/ 2026-01-21 12:26:11.284813 | controller | >f+++++++++ registries.conf.d/000-shortnames.conf 2026-01-21 12:26:11.284831 | controller | cd+++++++++ system-config/ 2026-01-21 12:26:11.284849 | controller | cd+++++++++ system-config/libvirt/ 2026-01-21 12:26:11.284866 | controller | >f+++++++++ system-config/libvirt/libvirt-admin.conf 2026-01-21 12:26:11.284884 | controller | >f+++++++++ system-config/libvirt/libvirt.conf 2026-01-21 12:26:11.796956 | controller | ok 2026-01-21 12:26:12.340533 | controller | ok 2026-01-21 12:26:12.417508 | 2026-01-21 12:26:12.417621 | TASK [Return artifact to Zuul] 2026-01-21 12:26:12.452545 | controller | skipping: Conditional result was False 2026-01-21 12:26:12.585548 | 2026-01-21 12:26:12.585651 | PLAY [Run ci/playbooks/collect-logs.yml on CRC host] 2026-01-21 12:26:12.606670 | 2026-01-21 12:26:12.606768 | TASK [Get kubelet journalctl logs] 2026-01-21 12:26:14.150485 | crc | Output suppressed because no_log was given 2026-01-21 12:26:14.165273 | 2026-01-21 12:26:14.165403 | TASK [Compress logs bigger than 2MB] 2026-01-21 12:26:14.474755 | crc | gzip: /var/home/core/zuul-output/logs/kubelet.log.gz already exists; not overwritten 2026-01-21 12:26:14.728378 | crc | ERROR 2026-01-21 12:26:14.728776 | crc | { 2026-01-21 12:26:14.728846 | crc | "delta": "0:00:00.009975", 2026-01-21 12:26:14.728889 | crc | "end": "2026-01-21 12:26:14.475901", 2026-01-21 12:26:14.728928 | crc | "msg": "non-zero return code", 2026-01-21 12:26:14.728965 | crc | "rc": 1, 2026-01-21 12:26:14.729001 | crc | "start": "2026-01-21 12:26:14.465926" 2026-01-21 12:26:14.729036 | crc | } 2026-01-21 12:26:14.729104 | crc | ERROR: Ignoring Errors 2026-01-21 12:26:14.744775 | 2026-01-21 12:26:14.744916 | TASK [Copy files from workspace on node] 2026-01-21 12:26:14.823224 | crc | ok 2026-01-21 12:26:14.842454 | 2026-01-21 12:26:14.842527 | TASK [fetch-output : Set log path for multiple nodes] 2026-01-21 12:26:14.894872 | crc | ok 2026-01-21 12:26:14.910799 | 2026-01-21 12:26:14.910989 | TASK [fetch-output : Set log path for single node] 2026-01-21 12:26:14.929430 | crc | skipping: Conditional result was False 2026-01-21 12:26:14.947579 | 2026-01-21 12:26:14.947768 | LOOP [fetch-output : Ensure local output dirs] 2026-01-21 12:26:15.232280 | crc -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/logs/crc" 2026-01-21 12:26:15.232529 | crc -> localhost | changed: All items complete 2026-01-21 12:26:15.232556 | 2026-01-21 12:26:15.476937 | crc -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/artifacts" 2026-01-21 12:26:15.748662 | crc -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/docs" 2026-01-21 12:26:15.771299 | 2026-01-21 12:26:15.771517 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-01-21 12:26:16.597417 | crc | changed: 2026-01-21 12:26:16.597865 | crc | .d..t...... ./ 2026-01-21 12:26:16.597931 | crc | >f+++++++++ kubelet.log 2026-01-21 12:26:17.147558 | crc | ok 2026-01-21 12:26:17.691668 | crc | ok 2026-01-21 12:26:17.764270 | 2026-01-21 12:26:17.764366 | PLAY RECAP 2026-01-21 12:26:17.764408 | controller | ok: 15 changed: 11 unreachable: 0 failed: 0 skipped: 7 rescued: 0 ignored: 1 2026-01-21 12:26:17.764433 | crc | ok: 5 changed: 3 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 1 2026-01-21 12:26:17.764452 | 2026-01-21 12:26:17.888622 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/collect-logs.yml@main] 2026-01-21 12:26:17.892325 | POST-RUN START: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/multinode-autohold.yml@main] 2026-01-21 12:26:18.483054 | 2026-01-21 12:26:18.483171 | PLAY [Run ci/playbooks/multinode-autohold.yml] 2026-01-21 12:26:18.579470 | 2026-01-21 12:26:18.579601 | TASK [Verify if "success" flag exists after successful tests execution] 2026-01-21 12:26:19.169087 | controller | ok 2026-01-21 12:26:19.191963 | 2026-01-21 12:26:19.192085 | TASK [Fetch existing autoholds from zuul] 2026-01-21 12:26:19.250049 | controller | ok 2026-01-21 12:26:19.300091 | 2026-01-21 12:26:19.300201 | TASK [krb_request : Perform the request] 2026-01-21 12:26:20.000973 | controller | ok 2026-01-21 12:26:20.098146 | 2026-01-21 12:26:20.098274 | TASK [Check if any autohold matches] 2026-01-21 12:26:20.194132 | controller | skipping: Conditional result was False 2026-01-21 12:26:20.290212 | 2026-01-21 12:26:20.290301 | PLAY RECAP 2026-01-21 12:26:20.290341 | controller | ok: 2 changed: 0 unreachable: 0 failed: 0 skipped: 1 rescued: 0 ignored: 0 2026-01-21 12:26:20.290362 | 2026-01-21 12:26:20.411288 | POST-RUN END RESULT_NORMAL: [untrusted : github.com/openstack-k8s-operators/ci-framework/ci/playbooks/multinode-autohold.yml@main] 2026-01-21 12:26:20.417028 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/print-machineconfigpool.yaml@master] 2026-01-21 12:26:21.029004 | 2026-01-21 12:26:21.029120 | PLAY [crc] 2026-01-21 12:26:21.057066 | 2026-01-21 12:26:21.057189 | TASK [Print machineconfigpool] 2026-01-21 12:26:22.020131 | crc | NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE 2026-01-21 12:26:22.028651 | crc | master rendered-master-f83a149c7490c3b0bf850df839837383 False False True 1 0 0 1 332d 2026-01-21 12:26:22.028729 | crc | worker rendered-worker-bf1bfcbe39747fb8c073115be50fcaab True False False 0 0 0 0 332d 2026-01-21 12:26:22.094133 | crc | changed 2026-01-21 12:26:22.127281 | 2026-01-21 12:26:22.127370 | PLAY RECAP 2026-01-21 12:26:22.127412 | crc | ok: 1 changed: 1 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-01-21 12:26:22.127434 | 2026-01-21 12:26:22.221239 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/print-machineconfigpool.yaml@master] 2026-01-21 12:26:22.225425 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/compress-crc-cloud-logs.yaml@master] 2026-01-21 12:26:22.792548 | 2026-01-21 12:26:22.792652 | PLAY [crc] 2026-01-21 12:26:22.818610 | 2026-01-21 12:26:22.818737 | TASK [Create compressed log tar gz file - crc-cloud logs] 2026-01-21 12:26:23.547838 | crc | tar: Removing leading `/' from member names 2026-01-21 12:26:23.552273 | crc | tar: /var/home/core/workdir/*log: Cannot stat: No such file or directory 2026-01-21 12:26:23.552311 | crc | tar: Exiting with failure status due to previous errors 2026-01-21 12:26:23.858575 | crc | ok 2026-01-21 12:26:23.873231 | 2026-01-21 12:26:23.873365 | TASK [Copy crc-cloud logs] 2026-01-21 12:26:24.259492 | crc | cp: cannot stat '/var/home/core/workdir/*log': No such file or directory 2026-01-21 12:26:24.440143 | crc | ok 2026-01-21 12:26:24.477574 | 2026-01-21 12:26:24.477674 | PLAY RECAP 2026-01-21 12:26:24.477749 | crc | ok: 2 changed: 0 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-01-21 12:26:24.477771 | 2026-01-21 12:26:24.601049 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/compress-crc-cloud-logs.yaml@master] 2026-01-21 12:26:24.605357 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/compress-logs.yaml@master] 2026-01-21 12:26:25.160876 | 2026-01-21 12:26:25.161060 | PLAY [all] 2026-01-21 12:26:25.250938 | 2026-01-21 12:26:25.251065 | TASK [Create compressed log tar gz file - ci-framework-data] 2026-01-21 12:26:25.994980 | controller | tar: Removing leading `/' from member names 2026-01-21 12:26:26.085891 | crc | tar: Removing leading `/' from member names 2026-01-21 12:26:26.085958 | crc | tar: /var/home/core/ci-framework-data: Cannot stat: No such file or directory 2026-01-21 12:26:26.089918 | crc | tar: Exiting with failure status due to previous errors 2026-01-21 12:26:26.092352 | crc | mv: cannot move '/var/home/core/ci-framework-data-crc-all-logs.tar.gz' to '/var/home/core/ci-framework-data/logs/ci-framework-data-crc-all-logs.tar.gz': No such file or directory 2026-01-21 12:26:26.096311 | crc | chown: cannot access '/var/home/core/ci-framework-data/logs/ci-framework-data-crc-all-logs.tar.gz': No such file or directory 2026-01-21 12:26:26.425022 | crc | ERROR 2026-01-21 12:26:26.425290 | crc | { 2026-01-21 12:26:26.425335 | crc | "delta": "0:00:00.021080", 2026-01-21 12:26:26.425363 | crc | "end": "2026-01-21 12:26:26.099650", 2026-01-21 12:26:26.425388 | crc | "msg": "non-zero return code", 2026-01-21 12:26:26.425411 | crc | "rc": 1, 2026-01-21 12:26:26.425434 | crc | "start": "2026-01-21 12:26:26.078570" 2026-01-21 12:26:26.425457 | crc | } 2026-01-21 12:26:26.425493 | crc | ERROR: Ignoring Errors 2026-01-21 12:26:28.785394 | controller | ok 2026-01-21 12:26:28.807965 | 2026-01-21 12:26:28.808041 | TASK [Create compressed log tar gz file - zuul-output] 2026-01-21 12:26:29.288356 | controller | tar: Removing leading `/' from member names 2026-01-21 12:26:29.371403 | crc | tar: Removing leading `/' from member names 2026-01-21 12:26:29.990010 | crc | ok 2026-01-21 12:26:30.110367 | controller | ok 2026-01-21 12:26:30.267186 | 2026-01-21 12:26:30.267288 | PLAY RECAP 2026-01-21 12:26:30.267325 | controller | ok: 2 changed: 0 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 0 2026-01-21 12:26:30.267350 | crc | ok: 2 changed: 0 unreachable: 0 failed: 0 skipped: 0 rescued: 0 ignored: 1 2026-01-21 12:26:30.267368 | 2026-01-21 12:26:30.368720 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/compress-logs.yaml@master] 2026-01-21 12:26:30.373068 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/crc/bootstraped-ci-network-cleanup.yml@master] 2026-01-21 12:26:30.950331 | 2026-01-21 12:26:30.950436 | PLAY [Run playbooks/crc/bootstraped-ci-network-cleanup.yml] 2026-01-21 12:26:31.025986 | 2026-01-21 12:26:31.026110 | TASK [Create openstack config dir] 2026-01-21 12:26:31.624609 | controller | ok 2026-01-21 12:26:31.647316 | 2026-01-21 12:26:31.647397 | TASK [Check for cleanup skip flag file presence] 2026-01-21 12:26:32.013160 | controller | ok 2026-01-21 12:26:32.039725 | 2026-01-21 12:26:32.039814 | TASK [Generate clouds config from cloud_secrets secret] 2026-01-21 12:26:33.022310 | controller | changed 2026-01-21 12:26:33.045005 | 2026-01-21 12:26:33.045077 | LOOP [Delete ports from each host] 2026-01-21 12:26:33.170541 | controller | ok: "crc" 2026-01-21 12:26:36.164517 | controller | ok: "controller" 2026-01-21 12:26:36.215657 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstraped-ci-network-cleanup-instance-ports.yml 2026-01-21 12:26:36.220925 | controller | included: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/trusted/project_0/review.rdoproject.org/config/playbooks/crc/bootstraped-ci-network-cleanup-instance-ports.yml 2026-01-21 12:26:36.250754 | 2026-01-21 12:26:36.250930 | TASK [Fetch port info] 2026-01-21 12:26:38.656967 | controller | admin_state_up: true 2026-01-21 12:26:38.657101 | controller | allowed_address_pairs: [] 2026-01-21 12:26:38.657322 | controller | binding_host_id: null 2026-01-21 12:26:38.657434 | controller | binding_profile: {} 2026-01-21 12:26:38.657560 | controller | binding_vif_details: {} 2026-01-21 12:26:38.657702 | controller | binding_vif_type: null 2026-01-21 12:26:38.657904 | controller | binding_vnic_type: normal 2026-01-21 12:26:38.658027 | controller | created_at: '2026-01-21T10:53:17Z' 2026-01-21 12:26:38.658138 | controller | data_plane_status: null 2026-01-21 12:26:38.658277 | controller | description: '' 2026-01-21 12:26:38.658403 | controller | device_id: 57887b03-108e-4b07-83a9-2cba1ffe7256 2026-01-21 12:26:38.658510 | controller | device_owner: compute:nova 2026-01-21 12:26:38.658611 | controller | device_profile: null 2026-01-21 12:26:38.658930 | controller | dns_assignment: 2026-01-21 12:26:38.659025 | controller | - fqdn: host-192-168-122-10.openstacklocal. 2026-01-21 12:26:38.659035 | controller | hostname: host-192-168-122-10 2026-01-21 12:26:38.659041 | controller | ip_address: 192.168.122.10 2026-01-21 12:26:38.659050 | controller | dns_domain: '' 2026-01-21 12:26:38.659138 | controller | dns_name: '' 2026-01-21 12:26:38.659287 | controller | extra_dhcp_opts: [] 2026-01-21 12:26:38.659567 | controller | fixed_ips: 2026-01-21 12:26:38.659676 | controller | - ip_address: 192.168.122.10 2026-01-21 12:26:38.659685 | controller | subnet_id: 29829789-a339-4200-85ef-2a8dc64a4ea5 2026-01-21 12:26:38.659694 | controller | hardware_offload_type: null 2026-01-21 12:26:38.659787 | controller | hints: '' 2026-01-21 12:26:38.659914 | controller | id: 8e10dcc4-a879-4901-80f6-7b4acc1dd1f6 2026-01-21 12:26:38.660063 | controller | ip_allocation: immediate 2026-01-21 12:26:38.660194 | controller | mac_address: fa:16:3e:0e:f1:50 2026-01-21 12:26:38.660314 | controller | name: crc-57887b03-108e-4b07-83a9-2cba1ffe7256 2026-01-21 12:26:38.660481 | controller | network_id: ccee4fed-07c7-4dcd-bc6b-76021b98eb53 2026-01-21 12:26:38.660593 | controller | numa_affinity_policy: null 2026-01-21 12:26:38.660776 | controller | port_security_enabled: false 2026-01-21 12:26:38.660900 | controller | project_id: 4b633c451ac74233be3721a3635275e5 2026-01-21 12:26:38.661000 | controller | propagate_uplink_status: null 2026-01-21 12:26:38.661098 | controller | resource_request: null 2026-01-21 12:26:38.661232 | controller | revision_number: 4 2026-01-21 12:26:38.661369 | controller | qos_network_policy_id: null 2026-01-21 12:26:38.661473 | controller | qos_policy_id: null 2026-01-21 12:26:38.661576 | controller | security_group_ids: [] 2026-01-21 12:26:38.661668 | controller | status: ACTIVE 2026-01-21 12:26:38.661773 | controller | tags: [] 2026-01-21 12:26:38.661870 | controller | trunk_details: null 2026-01-21 12:26:38.661961 | controller | trusted: null 2026-01-21 12:26:38.662072 | controller | updated_at: '2026-01-21T10:53:29Z' 2026-01-21 12:26:38.915508 | controller | changed 2026-01-21 12:26:38.947888 | 2026-01-21 12:26:38.948006 | TASK [Set port info yaml for further usage] 2026-01-21 12:26:39.084827 | controller | ok 2026-01-21 12:26:39.122671 | 2026-01-21 12:26:39.122854 | TASK [Detach port] 2026-01-21 12:26:42.282570 | controller | changed 2026-01-21 12:26:42.342045 | 2026-01-21 12:26:42.342167 | LOOP [Delete port association with trunk network] 2026-01-21 12:26:42.410888 | 2026-01-21 12:26:42.411067 | LOOP [Delete trunk sub ports (tagged ones)] 2026-01-21 12:26:42.507697 | 2026-01-21 12:26:42.507869 | TASK [Delete the trunk network itself] 2026-01-21 12:26:43.167249 | controller | skipping: Conditional result was False 2026-01-21 12:26:43.206128 | 2026-01-21 12:26:43.206264 | TASK [Remove the parent/trunk port itself] 2026-01-21 12:26:56.905163 | controller | changed 2026-01-21 12:26:56.976476 | 2026-01-21 12:26:56.976654 | TASK [Fetch port info] 2026-01-21 12:27:09.398942 | controller | admin_state_up: true 2026-01-21 12:27:09.399197 | controller | allowed_address_pairs: [] 2026-01-21 12:27:09.399527 | controller | binding_host_id: null 2026-01-21 12:27:09.399668 | controller | binding_profile: {} 2026-01-21 12:27:09.399919 | controller | binding_vif_details: {} 2026-01-21 12:27:09.399965 | controller | binding_vif_type: null 2026-01-21 12:27:09.400161 | controller | binding_vnic_type: normal 2026-01-21 12:27:09.400346 | controller | created_at: '2026-01-21T10:53:04Z' 2026-01-21 12:27:09.400481 | controller | data_plane_status: null 2026-01-21 12:27:09.400638 | controller | description: '' 2026-01-21 12:27:09.400814 | controller | device_id: edcda38a-13f5-423b-8fec-d9fb09ea9acf 2026-01-21 12:27:09.400941 | controller | device_owner: compute:nova 2026-01-21 12:27:09.401074 | controller | device_profile: null 2026-01-21 12:27:09.401511 | controller | dns_assignment: 2026-01-21 12:27:09.401644 | controller | - fqdn: host-192-168-122-11.openstacklocal. 2026-01-21 12:27:09.401657 | controller | hostname: host-192-168-122-11 2026-01-21 12:27:09.401665 | controller | ip_address: 192.168.122.11 2026-01-21 12:27:09.401676 | controller | dns_domain: '' 2026-01-21 12:27:09.401788 | controller | dns_name: '' 2026-01-21 12:27:09.401916 | controller | extra_dhcp_opts: [] 2026-01-21 12:27:09.402247 | controller | fixed_ips: 2026-01-21 12:27:09.402382 | controller | - ip_address: 192.168.122.11 2026-01-21 12:27:09.402397 | controller | subnet_id: 29829789-a339-4200-85ef-2a8dc64a4ea5 2026-01-21 12:27:09.402408 | controller | hardware_offload_type: null 2026-01-21 12:27:09.402510 | controller | hints: '' 2026-01-21 12:27:09.402702 | controller | id: 6989b816-3d0d-4682-887c-aa01e2e291eb 2026-01-21 12:27:09.402836 | controller | ip_allocation: immediate 2026-01-21 12:27:09.402989 | controller | mac_address: fa:16:3e:be:97:29 2026-01-21 12:27:09.403193 | controller | name: controller-edcda38a-13f5-423b-8fec-d9fb09ea9acf 2026-01-21 12:27:09.403349 | controller | network_id: ccee4fed-07c7-4dcd-bc6b-76021b98eb53 2026-01-21 12:27:09.403480 | controller | numa_affinity_policy: null 2026-01-21 12:27:09.403692 | controller | port_security_enabled: false 2026-01-21 12:27:09.403880 | controller | project_id: 4b633c451ac74233be3721a3635275e5 2026-01-21 12:27:09.404070 | controller | propagate_uplink_status: null 2026-01-21 12:27:09.404172 | controller | resource_request: null 2026-01-21 12:27:09.404356 | controller | revision_number: 4 2026-01-21 12:27:09.404496 | controller | qos_network_policy_id: null 2026-01-21 12:27:09.404654 | controller | qos_policy_id: null 2026-01-21 12:27:09.404850 | controller | security_group_ids: [] 2026-01-21 12:27:09.404962 | controller | status: ACTIVE 2026-01-21 12:27:09.405083 | controller | tags: [] 2026-01-21 12:27:09.405213 | controller | trunk_details: null 2026-01-21 12:27:09.405340 | controller | trusted: null 2026-01-21 12:27:09.405496 | controller | updated_at: '2026-01-21T10:53:16Z' 2026-01-21 12:27:09.690780 | controller | changed 2026-01-21 12:27:09.760017 | 2026-01-21 12:27:09.760133 | TASK [Set port info yaml for further usage] 2026-01-21 12:27:09.879229 | controller | ok 2026-01-21 12:27:09.901833 | 2026-01-21 12:27:09.901900 | TASK [Detach port] 2026-01-21 12:27:24.244136 | controller | changed 2026-01-21 12:27:24.277861 | 2026-01-21 12:27:24.277977 | LOOP [Delete port association with trunk network] 2026-01-21 12:27:24.378022 | 2026-01-21 12:27:24.378224 | LOOP [Delete trunk sub ports (tagged ones)] 2026-01-21 12:27:24.515583 | 2026-01-21 12:27:24.515750 | TASK [Delete the trunk network itself] 2026-01-21 12:27:25.135715 | controller | skipping: Conditional result was False 2026-01-21 12:27:25.188437 | 2026-01-21 12:27:25.188570 | TASK [Remove the parent/trunk port itself] 2026-01-21 12:27:39.575574 | controller | changed 2026-01-21 12:27:39.609512 | 2026-01-21 12:27:39.609630 | TASK [Dettach router from subnet] 2026-01-21 12:27:40.278891 | controller | skipping: Conditional result was False 2026-01-21 12:27:40.333726 | 2026-01-21 12:27:40.333869 | TASK [Delete router] 2026-01-21 12:27:40.973828 | controller | skipping: Conditional result was False 2026-01-21 12:27:41.057891 | 2026-01-21 12:27:41.058116 | TASK [Delete subnet] 2026-01-21 12:27:49.222453 | controller | changed 2026-01-21 12:27:49.276314 | 2026-01-21 12:27:49.276466 | TASK [Delete network] 2026-01-21 12:27:57.990005 | controller | changed 2026-01-21 12:27:58.041044 | 2026-01-21 12:27:58.041198 | TASK [Remove clouds.yml secret] 2026-01-21 12:27:58.449479 | controller | changed 2026-01-21 12:27:58.582136 | 2026-01-21 12:27:58.582239 | PLAY RECAP 2026-01-21 12:27:58.582281 | controller | ok: 16 changed: 10 unreachable: 0 failed: 0 skipped: 8 rescued: 0 ignored: 0 2026-01-21 12:27:58.582302 | 2026-01-21 12:27:58.683169 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/crc/bootstraped-ci-network-cleanup.yml@master] 2026-01-21 12:27:58.692180 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/ci-framework-rdo-base/dlrn/dlrn-report.yaml@master] 2026-01-21 12:27:59.254576 | 2026-01-21 12:27:59.254664 | PLAY [Report DLRN results after job run] 2026-01-21 12:27:59.339356 | 2026-01-21 12:27:59.339453 | TASK [Report job status to DLRN] 2026-01-21 12:27:59.402842 | controller | ok 2026-01-21 12:27:59.461923 | 2026-01-21 12:27:59.462024 | TASK [dlrn_report : Warning if DLRN is not running] 2026-01-21 12:27:59.506376 | controller | skipping: Conditional result was False 2026-01-21 12:27:59.540000 | 2026-01-21 12:27:59.540102 | TASK [dlrn_report : Install dlrnapi-client shyaml package] 2026-01-21 12:27:59.584695 | controller | skipping: Conditional result was False 2026-01-21 12:27:59.610912 | 2026-01-21 12:27:59.610991 | TASK [dlrn_report : Install kinit related package] 2026-01-21 12:27:59.655495 | controller | skipping: Conditional result was False 2026-01-21 12:27:59.698174 | 2026-01-21 12:27:59.698280 | TASK [dlrn_report : Install dlrn kerberos related packages] 2026-01-21 12:27:59.743620 | controller | skipping: Conditional result was False 2026-01-21 12:27:59.788915 | 2026-01-21 12:27:59.789015 | TASK [Set zuul-log-path fact] 2026-01-21 12:27:59.935940 | controller | skipping: Conditional result was False 2026-01-21 12:28:00.026941 | 2026-01-21 12:28:00.027067 | TASK [dlrn_report : Set the the value of cifmw_repo_setup_promotion when multiple dlrn tags are used] 2026-01-21 12:28:00.102298 | controller | skipping: Conditional result was False 2026-01-21 12:28:00.152943 | 2026-01-21 12:28:00.153049 | TASK [Get hash related data from repo_setup role] 2026-01-21 12:28:00.198734 | controller | skipping: Conditional result was False 2026-01-21 12:28:00.246827 | 2026-01-21 12:28:00.246943 | TASK [dlrn_report : Perform kinit for DLRN kerberos authentication] 2026-01-21 12:28:00.904215 | controller | skipping: Conditional result was False 2026-01-21 12:28:00.947491 | 2026-01-21 12:28:00.947663 | TASK [dlrn_report : Set empty value for dlrnapi password] 2026-01-21 12:28:01.025244 | controller | skipping: Conditional result was False 2026-01-21 12:28:01.077092 | 2026-01-21 12:28:01.077237 | TASK [dlrn_report : Report results to dlrn for the tested hash] 2026-01-21 12:28:01.738058 | controller | skipping: Conditional result was False 2026-01-21 12:28:01.928625 | 2026-01-21 12:28:01.928764 | PLAY RECAP 2026-01-21 12:28:01.928808 | controller | ok: 0 changed: 0 unreachable: 0 failed: 0 skipped: 10 rescued: 0 ignored: 0 2026-01-21 12:28:01.928831 | 2026-01-21 12:28:02.051480 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/ci-framework-rdo-base/dlrn/dlrn-report.yaml@master] 2026-01-21 12:28:02.056507 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/base-minimal/post-ssh.yaml@master] 2026-01-21 12:28:02.703120 | 2026-01-21 12:28:02.703230 | PLAY [all] 2026-01-21 12:28:02.792026 | 2026-01-21 12:28:02.792165 | TASK [include_role : fetch-output] 2026-01-21 12:28:02.872845 | controller | ok 2026-01-21 12:28:02.879622 | crc | ok 2026-01-21 12:28:02.939794 | 2026-01-21 12:28:02.939964 | TASK [fetch-output : Set log path for multiple nodes] 2026-01-21 12:28:03.014557 | controller | ok 2026-01-21 12:28:03.021711 | crc | ok 2026-01-21 12:28:03.043814 | 2026-01-21 12:28:03.043880 | TASK [fetch-output : Set log path for single node] 2026-01-21 12:28:03.076606 | controller | skipping: Conditional result was False 2026-01-21 12:28:03.097717 | crc | skipping: Conditional result was False 2026-01-21 12:28:03.153073 | 2026-01-21 12:28:03.153201 | LOOP [fetch-output : Ensure local output dirs] 2026-01-21 12:28:03.630333 | controller -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/logs/controller" 2026-01-21 12:28:03.660638 | crc -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/logs/crc" 2026-01-21 12:28:03.863020 | controller -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/artifacts" 2026-01-21 12:28:03.888951 | crc -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/artifacts" 2026-01-21 12:28:04.076065 | controller -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/docs" 2026-01-21 12:28:04.106489 | crc -> localhost | ok: "/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/docs" 2026-01-21 12:28:04.106640 | crc -> localhost | ok: All items complete 2026-01-21 12:28:04.106665 | 2026-01-21 12:28:04.137279 | 2026-01-21 12:28:04.137398 | LOOP [fetch-output : Collect logs, artifacts and docs] 2026-01-21 12:28:04.879392 | controller | changed: 2026-01-21 12:28:04.879603 | controller | .d..t...... ./ 2026-01-21 12:28:04.879640 | controller | >f+++++++++ zuul-output-controller-all-logs.tar.gz 2026-01-21 12:28:04.886525 | crc | changed: 2026-01-21 12:28:04.886574 | crc | .d..t...... ./ 2026-01-21 12:28:04.886599 | crc | >f+++++++++ crc-cloud-workdir-crc-all-logs.tar.gz 2026-01-21 12:28:04.886619 | crc | >f+++++++++ zuul-output-crc-all-logs.tar.gz 2026-01-21 12:28:04.886638 | crc | cd+++++++++ crc-cloud/ 2026-01-21 12:28:05.316860 | controller | ok 2026-01-21 12:28:05.367725 | crc | ok 2026-01-21 12:28:05.781772 | controller | ok 2026-01-21 12:28:05.926470 | crc | ok 2026-01-21 12:28:05.926758 | crc | changed: All items complete 2026-01-21 12:28:05.926828 | 2026-01-21 12:28:06.042047 | 2026-01-21 12:28:06.042186 | TASK [include_role : fetch-output-openshift] 2026-01-21 12:28:06.059713 | controller | skipping: Conditional result was False 2026-01-21 12:28:06.070666 | crc | skipping: Conditional result was False 2026-01-21 12:28:06.095537 | 2026-01-21 12:28:06.095612 | LOOP [merge-output-to-logs : Move artifacts and docs to logs dir] 2026-01-21 12:28:06.488363 | controller -> localhost | ok: Item: artifacts Runtime: 0:00:00.013857 2026-01-21 12:28:06.789121 | controller -> localhost | ok: Item: docs Runtime: 0:00:00.008608 2026-01-21 12:28:07.016070 | 2026-01-21 12:28:07.016207 | PLAY [all] 2026-01-21 12:28:07.098722 | 2026-01-21 12:28:07.098880 | TASK [remove-build-sshkey : Remove the build SSH key from all nodes] 2026-01-21 12:28:07.861864 | controller | changed 2026-01-21 12:28:07.948059 | crc | changed 2026-01-21 12:28:08.148404 | 2026-01-21 12:28:08.148480 | PLAY RECAP 2026-01-21 12:28:08.148521 | controller | ok: 5 changed: 3 unreachable: 0 failed: 0 skipped: 2 rescued: 0 ignored: 0 2026-01-21 12:28:08.148550 | crc | ok: 4 changed: 2 unreachable: 0 failed: 0 skipped: 2 rescued: 0 ignored: 0 2026-01-21 12:28:08.148570 | 2026-01-21 12:28:08.259120 | POST-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/base-minimal/post-ssh.yaml@master] 2026-01-21 12:28:08.263652 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/base-minimal/post-logs.yaml@master] 2026-01-21 12:28:11.011939 | 2026-01-21 12:28:11.012054 | PLAY [localhost] 2026-01-21 12:28:11.029345 | 2026-01-21 12:28:11.029422 | TASK [generate-zuul-manifest : Generate Zuul manifest] 2026-01-21 12:28:11.477496 | localhost | changed 2026-01-21 12:28:11.482095 | 2026-01-21 12:28:11.482226 | TASK [generate-zuul-manifest : Return Zuul manifest URL to Zuul] 2026-01-21 12:28:11.510238 | localhost | ok 2026-01-21 12:28:11.518814 | 2026-01-21 12:28:11.518878 | TASK [add-fileserver : Create SSH private key tempfile] 2026-01-21 12:28:11.891328 | localhost | changed 2026-01-21 12:28:11.896634 | 2026-01-21 12:28:11.896723 | TASK [add-fileserver : Create SSH private key from secret] 2026-01-21 12:28:12.640889 | localhost | changed 2026-01-21 12:28:12.646041 | 2026-01-21 12:28:12.646103 | TASK [add-fileserver : Add fileserver ssh key] 2026-01-21 12:28:13.098155 | localhost | Identity added: /var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/tmp/ansible.az5gwzkm (/var/lib/zuul/builds/4d9066f27a1f41f7af2512c243527c41/work/tmp/ansible.az5gwzkm) 2026-01-21 12:28:13.098341 | localhost | ok: Runtime: 0:00:00.010717 2026-01-21 12:28:13.104319 | 2026-01-21 12:28:13.104381 | TASK [add-fileserver : Remove SSH private key from disk] 2026-01-21 12:28:13.427169 | localhost | ok: Runtime: 0:00:00.022522 2026-01-21 12:28:13.434099 | 2026-01-21 12:28:13.434166 | TASK [add-fileserver : Add fileserver to inventory] 2026-01-21 12:28:13.613742 | localhost | changed 2026-01-21 12:28:13.618767 | 2026-01-21 12:28:13.618835 | TASK [add-fileserver : Add fileserver server to known hosts] 2026-01-21 12:28:14.134518 | localhost | changed 2026-01-21 12:28:14.160784 | 2026-01-21 12:28:14.160883 | PLAY [localhost] 2026-01-21 12:28:14.173345 | 2026-01-21 12:28:14.173415 | TASK [Generate bulk log download script] 2026-01-21 12:28:14.191247 | localhost | ok 2026-01-21 12:28:14.204562 | 2026-01-21 12:28:14.204635 | TASK [local-log-download : Check API endpoint is defined] 2026-01-21 12:28:14.235445 | localhost | ok: All assertions passed 2026-01-21 12:28:14.240124 | 2026-01-21 12:28:14.240186 | TASK [local-log-download : Create download script] 2026-01-21 12:28:14.692626 | localhost -> localhost | changed 2026-01-21 12:28:14.701515 | 2026-01-21 12:28:14.701579 | TASK [Register quick-download link] 2026-01-21 12:28:14.719600 | localhost | ok 2026-01-21 12:28:14.736132 | 2026-01-21 12:28:14.736190 | PLAY [logserver.rdoproject.org] 2026-01-21 12:28:14.748758 | 2026-01-21 12:28:14.748827 | TASK [Set zuul-log-path fact] 2026-01-21 12:28:14.765039 | logserver.rdoproject.org | ok 2026-01-21 12:28:14.779639 | 2026-01-21 12:28:14.779782 | TASK [set-zuul-log-path-fact : Set log path for a build] 2026-01-21 12:28:14.807468 | logserver.rdoproject.org | ok 2026-01-21 12:28:14.813177 | 2026-01-21 12:28:14.813241 | TASK [upload-logs : Create log directories] 2026-01-21 12:28:15.760084 | logserver.rdoproject.org | changed 2026-01-21 12:28:15.771443 | 2026-01-21 12:28:15.771577 | TASK [upload-logs : Ensure logs are readable before uploading] 2026-01-21 12:28:16.085423 | logserver.rdoproject.org -> localhost | ok: Runtime: 0:00:00.039792 2026-01-21 12:28:16.096109 | 2026-01-21 12:28:16.096259 | TASK [upload-logs : Upload logs to log server] 2026-01-21 12:28:18.227741 | logserver.rdoproject.org | Output suppressed because no_log was given 2026-01-21 12:28:18.232922 | 2026-01-21 12:28:18.232984 | LOOP [upload-logs : Compress console log and json output] 2026-01-21 12:28:18.278661 | logserver.rdoproject.org | skipping: Conditional result was False 2026-01-21 12:28:18.282670 | logserver.rdoproject.org | skipping: Conditional result was False 2026-01-21 12:28:18.298062 | 2026-01-21 12:28:18.298190 | LOOP [upload-logs : Upload compressed console log and json output] 2026-01-21 12:28:18.345802 | logserver.rdoproject.org | skipping: Conditional result was False 2026-01-21 12:28:18.346077 | 2026-01-21 12:28:18.349426 | logserver.rdoproject.org | skipping: Conditional result was False 2026-01-21 12:28:18.361342 | 2026-01-21 12:28:18.361432 | LOOP [upload-logs : Upload console log and json output]